-- Logs begin at Sun 2022-08-07 00:10:30 UTC, end at Sun 2022-08-07 01:58:20 UTC. -- Aug 07 01:00:44.378144 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Aug 07 01:00:44.387723 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Aug 07 01:00:44.389797 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Aug 7 01:00:44 2022] *** Aug 07 01:00:44.389797 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 07 01:00:44.389797 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 07 01:00:44.389797 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: nodename: dv-cinder-master-3-iscsi Aug 07 01:00:44.389797 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: machine: x86_64 Aug 07 01:00:44.389797 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: clock source: unix Aug 07 01:00:44.389797 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: pcre jit disabled Aug 07 01:00:44.389797 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: detected number of CPU cores: 4 Aug 07 01:00:44.389797 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: current working directory: / Aug 07 01:00:44.389797 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: detected binary path: /usr/bin/uwsgi-core Aug 07 01:00:44.389797 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: your processes number limit is 47680 Aug 07 01:00:44.389797 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: your memory page size is 4096 bytes Aug 07 01:00:44.390633 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: detected max file descriptor number: 2048 Aug 07 01:00:44.390633 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: lock engine: pthread robust mutexes Aug 07 01:00:44.390633 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: thunder lock: enabled Aug 07 01:00:44.390633 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Aug 07 01:00:44.390633 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 07 01:00:44.429706 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: Python main interpreter initialized at 0x5572c17369c0 Aug 07 01:00:44.429706 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: python threads support enabled Aug 07 01:00:44.429706 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: your server socket listen backlog is limited to 100 connections Aug 07 01:00:44.429706 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: your mercy for graceful operations on workers is 90 seconds Aug 07 01:00:44.430846 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: mapped 403077 bytes (393 KB) for 2 cores Aug 07 01:00:44.430846 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: *** Operational MODE: preforking *** Aug 07 01:00:44.431122 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: *** uWSGI is running in multiple interpreter mode *** Aug 07 01:00:44.431122 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: spawned uWSGI master process (pid: 103440) Aug 07 01:00:44.434608 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: spawned uWSGI worker 1 (pid: 103441, cores: 1) Aug 07 01:00:44.434608 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: spawned uWSGI worker 2 (pid: 103442, cores: 1) Aug 07 01:00:44.434608 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 07 01:00:44.431379 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Aug 07 01:00:45.113075 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 07 01:00:45.187717 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 07 01:00:46.303702 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 01:00:46.303702 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: from cryptography.utils import int_from_bytes Aug 07 01:00:46.305381 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 01:00:46.305381 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: from cryptography.utils import int_from_bytes Aug 07 01:00:46.315695 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 01:00:46.315695 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: from cryptography.utils import int_from_bytes Aug 07 01:00:46.316465 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 01:00:46.316465 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: from cryptography.utils import int_from_bytes Aug 07 01:00:48.875280 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 07 01:00:48.876048 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 07 01:00:48.876414 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 07 01:00:48.876713 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 07 01:00:48.877004 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 07 01:00:48.877401 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.877728 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.878051 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.878376 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.878705 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.879014 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.879336 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.879656 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.879985 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.880265 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.880582 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.880912 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.881242 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.881548 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.881860 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.882181 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.882485 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.882840 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.883146 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.883649 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.885226 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.885226 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.885226 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.885226 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.885655 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.885655 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.885977 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.886274 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.886598 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.886987 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.887456 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.887779 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.888102 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.888472 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.888894 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.889202 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.889511 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.889829 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.890164 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.890470 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.890797 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.891117 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.891445 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.891760 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.892061 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.892394 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.892710 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.893036 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.893341 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.893645 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.893895 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.894200 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.894525 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.894845 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.895165 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.895415 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.895664 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.895980 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.896324 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.896634 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.896948 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.897341 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.897543 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.897847 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.898281 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.898982 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.899286 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.899606 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.899931 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.900297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.900575 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.900920 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.901399 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.901606 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.901897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.902178 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.902495 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.902803 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.903175 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.903432 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.903737 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.904041 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.904368 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.904693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.905028 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.905331 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.905581 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.905877 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.906232 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.906491 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.906792 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.907162 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.907577 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.907745 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.907978 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.908342 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.908607 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.908945 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.909211 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.909522 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.909809 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.910249 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.910412 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.910719 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.911035 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.911338 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.911640 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.912090 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.912371 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.912589 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.912869 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.913165 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.913483 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.913785 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.914699 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.914957 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.915259 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.915560 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.915875 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.916172 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.916498 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.916803 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.917110 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.917579 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.918052 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.918052 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.918352 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.918549 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.918847 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.919146 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.919443 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.919758 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.920078 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.920411 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.920712 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.921009 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.921325 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.921662 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.921992 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.922357 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.922636 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.922968 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.923294 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.923610 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.923919 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.924230 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.924585 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.924894 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.925207 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.925508 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.925823 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.926139 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.926469 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.926721 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 07 01:00:48.927404 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.927617 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 07 01:00:48.927783 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.927783 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.928404 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 07 01:00:48.928569 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.928569 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.928844 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 07 01:00:48.928844 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 07 01:00:48.929231 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.929231 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.929589 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.929757 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.929906 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.930047 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.930186 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.930333 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.930485 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.930636 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.930786 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.930926 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.931077 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.931216 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.931370 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.931518 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.931658 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.931838 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.932029 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.932173 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.932641 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.932804 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.932958 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.932958 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.933246 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.933246 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.933246 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.933946 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.934095 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.934235 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.934374 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.934523 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.934666 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.934813 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.934953 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.935099 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.935239 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.935402 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.935542 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.935682 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.935828 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.935828 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.936104 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.936324 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.936529 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.936676 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.936816 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.936962 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.937131 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.937280 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.937426 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.937565 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.937704 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.937843 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.937997 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.938145 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.938551 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.938701 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.938842 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.939241 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.939393 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.939554 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.939695 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.939844 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.939997 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.940144 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.940304 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.940467 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.940611 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.940765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.940907 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.941052 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.941052 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.941315 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.941466 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.941614 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.941760 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.941898 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.942037 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.942176 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.942322 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.942469 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.942637 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.942917 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.943066 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.943205 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.943367 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.943516 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.943656 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.943795 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.943954 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.944308 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.944472 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.944627 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.944776 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.944927 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.945067 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.945693 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.945844 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.946007 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.946160 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.946300 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.946439 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.946577 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.946577 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.946855 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.946994 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.947140 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.947290 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.947442 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.947590 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.947730 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.947730 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.948003 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.948003 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.948461 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.948846 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.948996 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.949143 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.949285 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.949423 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.949583 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.949583 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.949860 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.950006 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.950147 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.950294 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.950450 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.950595 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.950742 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.950882 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.951027 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.951255 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.951410 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.951550 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.951698 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.951863 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.952003 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.952151 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.952315 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.952461 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.952610 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.952751 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.952909 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.953187 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.953354 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.953504 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.953653 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.953793 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.953946 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.954095 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.954236 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.954506 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.954651 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.954790 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.954929 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.955078 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.955219 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.955358 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.955498 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.955638 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.955875 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.956017 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.956165 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.956369 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.956510 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.956650 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.956801 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.956941 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.957081 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.957230 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.957370 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.957509 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.957663 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.957816 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.957816 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.958081 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.958081 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.958351 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.958499 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.958646 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.959057 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.959213 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.959213 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.959496 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.959646 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.959787 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.959938 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.960083 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.960223 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.960401 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.960559 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.960700 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.960842 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.960842 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.961646 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.961796 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.961937 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.962075 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.962215 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.962374 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.962513 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.962661 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.962801 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.962940 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.963079 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.963352 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.963500 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.963650 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.963650 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.963912 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.964051 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.964190 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.964376 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.964535 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.964784 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.964928 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.965068 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.965215 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.965364 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.965517 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.965665 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.965804 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.965953 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.966095 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.966257 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.966398 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.966536 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.966681 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.966828 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.966967 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.967116 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.967257 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.967403 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.967543 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.967682 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.967822 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.967970 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.968121 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.968335 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.968484 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.968624 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.968763 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.968929 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.969303 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.969458 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.969612 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.969754 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.969903 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.970043 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.970182 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.970333 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.970333 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.970610 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.970758 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.970899 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.971037 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.971175 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.971323 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.971463 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.971610 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.971759 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.971907 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.972047 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.972200 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.972371 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.972533 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.972683 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.972822 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.972960 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.973099 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.973238 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.973530 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.973685 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.973826 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.973965 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.974102 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.974262 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.974420 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.974562 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.974701 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.974964 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:00:48.975111 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.975261 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.975415 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.975556 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.975702 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.975859 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.975999 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.976159 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.976417 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.976417 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.976787 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.976931 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.977088 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.977088 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.977353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.977353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.977668 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.977807 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.977955 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.978097 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.978255 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.978395 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.978534 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.978673 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.978821 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.978961 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.979100 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.979487 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.979644 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.979794 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.979794 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.980070 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.980227 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.980410 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.980570 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.980721 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.980862 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.981002 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.981148 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.981301 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.981449 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.981598 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.981745 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.981887 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.982026 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.982026 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.982300 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.982449 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.982589 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.982735 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.982887 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.983027 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.983181 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.983330 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.983469 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.983757 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.983903 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.984051 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.984319 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.984468 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.984609 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.984749 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.984896 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.985163 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.985323 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.985478 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.985620 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.985759 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.985907 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.986055 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.986214 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.986356 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.986494 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.986642 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.986791 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.986944 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.987084 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.987237 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.987237 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.987501 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.987640 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.987779 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.987917 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.988082 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.988333 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.988487 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.988627 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.988776 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.988915 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.989068 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.989209 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.989376 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.989752 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.989906 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.990062 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.990218 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.990375 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.990537 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.990688 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.990841 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.990997 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.991144 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.991291 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.991436 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.991581 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.991734 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.991889 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.992084 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.992348 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.992348 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.992666 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.992814 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.992973 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.993120 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.993266 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.993413 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.993558 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.993732 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.994009 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.994170 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.994328 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.994476 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.994643 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.994800 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.994956 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.995102 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.995373 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.995528 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.995674 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.995828 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.995984 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.996139 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.996310 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.996465 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.996623 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.996789 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.996936 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.997094 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.997241 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.997386 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.997542 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.997689 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.997843 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.997989 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.998136 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.998289 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.998444 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.998603 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.998748 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.998892 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.999037 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.999189 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.999344 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.999499 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:48.999668 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.000047 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.000199 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.000436 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.000608 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.000758 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.000916 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.001061 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.001214 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.001370 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.001516 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.001661 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.001814 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.001961 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.001961 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.002251 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.002398 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.002542 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.002699 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.002860 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.003016 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.003178 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.003324 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.003476 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.003623 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.003776 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.003931 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.004253 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.004413 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.004560 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.004705 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.004865 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.005023 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.005178 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.005325 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.005601 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.005762 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.005912 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.006067 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.006215 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.006360 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.006506 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.006652 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.006798 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.006968 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.007116 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.007261 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.007414 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.007560 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.007560 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.008429 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.008429 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.008719 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.008719 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.008996 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.009152 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.009312 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.009472 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.009617 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.009764 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.010160 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.010334 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.010483 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.010630 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.010776 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.010921 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.011086 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.011245 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.011391 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.011538 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.011538 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.011829 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.011974 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.012211 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.012395 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.012543 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.012689 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.012834 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.012996 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.013143 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.013290 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.013435 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.013589 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.013747 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.013902 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.014048 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.014333 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.014492 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.014639 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.014784 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.014939 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.015086 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.015232 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.015379 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.015534 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.015820 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.015979 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.016126 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.016308 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.016464 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.016610 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.016770 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.016924 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.017070 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.017215 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.017369 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.017516 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.017672 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.017837 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.017984 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.018129 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.018274 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.018433 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.018593 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.018739 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.018885 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.019030 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.019175 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.019328 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.019491 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.019646 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.019813 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.019960 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.020464 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.020635 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.020842 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.020995 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.021153 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.021298 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.021444 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.021600 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.021600 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.021889 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.022035 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.022192 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.022339 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.022497 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.022659 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.022807 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.022953 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.023098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.023244 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.023962 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.024141 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.024317 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.024624 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.024774 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.024932 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.025087 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.025243 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.025470 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.025626 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.025928 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.026094 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.026277 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.026468 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.026617 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.026764 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.026925 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.027099 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.027247 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.027400 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.027547 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.027693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.027847 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.028015 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.028162 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.028338 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.028487 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.028657 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.028851 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.029080 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.029247 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.029394 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.029539 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.029702 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.029907 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.030096 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.030391 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.030674 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.030892 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.031097 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.031314 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.031466 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.031623 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.031783 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.031961 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.032120 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.032120 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.032482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.032658 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.032861 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.033020 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.033020 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.033318 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.033464 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.033464 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.033748 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.033906 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.033906 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.034194 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.034340 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.034340 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.034774 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.034922 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.035085 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.035234 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.035234 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.035546 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.035739 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.036054 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.036221 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.036221 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.036604 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.036756 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.036903 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.036903 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.037188 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.037188 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.037477 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.037630 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.037777 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.037948 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.038185 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.038408 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.038644 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.038921 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.039644 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.039891 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.040082 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.040313 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.040604 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.040890 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.041108 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.041287 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.041443 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.041618 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.041766 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.041911 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.042056 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.042211 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.042358 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.042510 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.042665 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.042836 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.043057 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.043210 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.043365 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.043524 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.043673 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.043830 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.044003 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.044003 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.044307 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.044307 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.044591 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.044865 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.045105 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.045255 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.045411 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.045558 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.045775 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.045950 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.045950 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.046376 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.046524 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.046670 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.046826 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.046986 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.047140 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.047286 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.047438 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.047438 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.047713 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.047912 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.048081 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.048356 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.048356 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.048644 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.048791 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.048938 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.049097 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.049245 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.049391 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.049536 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.049685 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.049833 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.050006 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.050154 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.050298 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.050443 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.050608 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.050878 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.051154 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.051361 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.051519 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.051667 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.051815 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.051960 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.052127 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.052308 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.052470 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.052618 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.052765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.052919 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.053066 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.053212 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.053358 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.053513 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.053659 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.053814 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.053970 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.054128 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.054274 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.054419 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.054419 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.055472 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.055627 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.055774 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.055920 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.056066 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.056283 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.056440 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.056440 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.056727 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.056887 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.057033 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.057188 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.057343 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.057497 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.057643 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.057789 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.057934 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.058088 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.058234 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.058380 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.058525 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.058681 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.058827 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.058995 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.059164 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.059310 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.059457 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.059602 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.059757 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.059905 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.060050 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.060220 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.060415 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.060574 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.060743 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.061038 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.061328 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.061484 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.061631 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.061777 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.061937 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.062085 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.062231 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.062377 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.062377 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.062671 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.062828 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.062989 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.063135 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.063280 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.063425 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.063585 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.063731 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.064124 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.064326 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.064482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.064630 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.064788 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.064944 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.065099 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.065257 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.065403 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.065678 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.065841 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.065996 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.066144 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.066289 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.066446 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.066594 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.066754 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.066901 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.067055 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.067209 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.067367 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.067522 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.067675 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.067823 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.067968 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.068113 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.068283 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.068283 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.068592 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.068592 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.068870 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.069015 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.069178 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.069340 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.069488 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.069633 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.069778 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.069931 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.070078 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.070273 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.070273 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.070571 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.070740 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.070740 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.071059 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.071345 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.071594 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.071756 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.071756 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.072047 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.072192 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.072192 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.072564 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.072712 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.072874 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.073022 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.073022 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.073324 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.073481 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.073481 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.073765 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.073911 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.074058 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.074212 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.074212 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.074496 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.074650 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.074650 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.074928 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.075094 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.075094 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.075379 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.075526 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.075526 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.075928 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.076089 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.076269 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.076423 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.076423 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.076777 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.076953 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.076953 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.077258 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.077404 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.077404 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.077728 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.077887 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.077887 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.078164 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.078311 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.078457 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.078602 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.078602 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.078889 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.079056 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.079206 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.079361 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.079361 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.079657 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.079806 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.079806 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.080092 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.080284 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.080284 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.080585 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.080732 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.080732 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.081019 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.081178 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.081178 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.081604 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.081845 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.081845 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.082127 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.082274 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.082274 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.082570 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.082717 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.082866 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.083019 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.083019 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.083358 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.083508 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.083508 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.083784 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.083930 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.084077 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.084281 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.084442 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.084591 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.084748 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.084904 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.084904 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.085204 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.085352 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.085352 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.085631 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.085804 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.085804 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.086257 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.086406 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.086406 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.086684 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.086838 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.086838 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.087145 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.087293 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.087293 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.087580 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.087726 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.087726 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.088014 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.088169 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.088169 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.088499 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.088658 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.088815 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.088963 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.088963 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.089251 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.089398 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.089398 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.089700 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.089896 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.090049 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.090195 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.090342 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.090573 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.090730 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.090879 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.090879 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.091188 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.091336 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.091336 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.091745 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.092009 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.092009 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.092344 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.092509 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.092665 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.092813 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.092813 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.093090 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.093235 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.093409 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.093565 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.093565 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.093843 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.093989 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.093989 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.094281 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.094428 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.094575 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.094720 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.094866 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.095011 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.095178 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.095325 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.095479 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.095682 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.095886 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.096069 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.096402 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.096557 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.096716 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.096915 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.097078 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.097241 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.097517 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.097899 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.098112 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.098263 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.098409 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.098636 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.098785 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.098938 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.099093 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.099242 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.099397 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.099581 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.099735 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.099895 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.100176 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.100365 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.100527 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.100796 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.100949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.101096 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.101987 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.102178 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.102293 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.102535 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.102677 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.102876 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.103187 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.103332 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.103476 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.103665 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.103787 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.104076 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.104223 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.104403 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.104700 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.104840 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.105026 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.105336 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.105477 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.105628 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.105844 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.106043 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.106353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.106435 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.106673 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.106979 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.107133 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.107291 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.107543 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.107706 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.107913 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.108085 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.108304 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.108548 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.108716 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.108883 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.109164 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.109320 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.109501 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.109828 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.110058 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.110270 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.110515 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.110684 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.110834 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.111025 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.111170 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.111293 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.111605 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.111751 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.111893 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.112198 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.112622 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.112622 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.112878 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.113014 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.113127 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.113360 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.113479 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.113662 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.114106 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.114217 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.114299 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.114535 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.114622 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.114833 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.115129 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.115215 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.115428 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.115734 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.115855 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.116032 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.116334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.116452 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.116644 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.116932 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.117102 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.117780 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.118081 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.118282 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.118508 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.118744 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.118932 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.119083 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.119259 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.119450 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.119675 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.119904 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.120159 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.120427 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.120600 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.120749 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.120959 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.121110 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.121281 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.121561 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.121716 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.121891 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.122326 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.122723 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.122949 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.122949 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.123391 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.123621 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.123621 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.124072 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.124415 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.124415 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.124870 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.125099 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.125389 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.125631 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.125631 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.126009 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.126182 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.126182 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.126552 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.126952 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.126952 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.127424 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.127632 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.127632 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.128019 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.128169 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.128169 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.128487 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.128703 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.128926 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.129182 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.129182 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.129612 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.129840 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.129840 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.130195 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.130407 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.130407 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.130825 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.130987 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.130987 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.131357 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.131576 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.131796 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.132023 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.132023 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.132602 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.133004 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.133004 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.133416 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.133661 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.133661 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.134075 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.134305 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.134463 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.134631 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.134631 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.134987 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.135235 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.135235 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 07 01:00:49.135691 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.135691 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.135691 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.135691 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.136492 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.136492 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.137158 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.137505 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.138068 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.138619 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.139182 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.139827 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.140690 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.140984 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.141604 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.142208 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.142724 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.143164 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.143741 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.144322 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.144897 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.145515 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.146098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.146682 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.147253 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.147811 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.148439 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.149007 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.149579 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.150137 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.150643 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.151280 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.151809 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.152198 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.152611 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.153085 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.153594 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.154131 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.154563 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.155147 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.155664 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.156263 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.156778 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.157348 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.157899 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.158457 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.158998 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.159364 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.159708 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.160277 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.160823 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.161367 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.161903 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.162461 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.163004 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.164522 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.165112 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.165674 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.166210 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.166769 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.167328 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.167893 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.168486 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.169047 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.169611 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.170157 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.170863 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.171306 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.171881 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.172522 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.173072 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.173644 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.174270 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.174778 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.175408 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.175967 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.176554 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.177124 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.177670 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.178245 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.178802 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.179382 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.180535 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.181100 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.181688 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.182259 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.182818 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.183434 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.184149 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.184613 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.185248 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.185756 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.186312 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.186873 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.187952 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.188482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.189310 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.189901 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.190492 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.191024 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.191584 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.192151 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.192687 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.193223 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.193801 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.194356 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.194923 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.195541 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.196110 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.196705 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.197299 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.197879 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.198924 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.199559 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.200147 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.200758 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.201347 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.201922 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.202487 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.203083 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.203661 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.204226 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.204798 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.205305 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.205831 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.206362 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.206939 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.207498 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.208042 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.208628 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.209174 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.209743 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.210297 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.210838 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.211386 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.211953 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.212547 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.213087 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.213667 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.214219 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.214811 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.215366 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.215922 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.216512 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.217092 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.217658 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.218220 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.218791 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.219498 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.220050 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.220716 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.221311 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.221907 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.222523 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.223117 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.223714 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.224387 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.224954 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.225988 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.227308 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.227897 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.228560 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.229154 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.229694 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.230231 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.230779 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.231381 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.231937 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.232526 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.233097 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.233666 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.234230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.234796 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.235380 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.235954 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.236569 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.237120 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.237702 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.238250 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.238828 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.239370 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.239964 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.240649 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.241211 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.241771 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.242888 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.243460 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.244015 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.244616 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.245182 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.245718 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.246314 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.246926 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.247482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.248022 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.248833 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.249390 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.249967 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.250551 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.251119 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.251705 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.252328 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.252871 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.253453 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.254012 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.254615 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.255212 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.255777 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.256395 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.256961 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.257550 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.258171 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.258769 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.259535 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.259939 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.260530 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.261100 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.261646 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.262172 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.262699 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.263254 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.263774 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.264365 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.264900 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.265439 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.266005 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.266571 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.267116 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.267674 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.268297 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.268817 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.269372 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.269924 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.270486 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.271051 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.271657 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.272283 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.272868 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.273497 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.274071 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.274710 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.275285 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.276147 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.276531 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.277098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.277659 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.278259 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.278796 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.279357 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.279925 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.280535 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.281076 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.281615 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.282169 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.283082 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.283326 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.283866 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.284440 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.285145 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.285558 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.286113 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.286712 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.287262 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.287855 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.288407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.289480 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.290027 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.290530 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.291037 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.291568 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.292115 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.292730 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.292911 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO dbcounter [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Registered counter for database nova_api Aug 07 01:00:49.293324 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.293919 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.294495 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.295055 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.295615 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.296292 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.296799 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.297400 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.297992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.298608 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.299157 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.303263 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:00:49.303263 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 07 01:00:49.316702 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_db.sqlalchemy.engines [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103442) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:00:49.324675 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writer thread running {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:00:49.389772 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.context [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dfea601e-2ab8-4cc9-8a98-b642673a3d20(cell1) {{(pid=103442) load_cells /opt/stack/nova/nova/context.py:464}} Aug 07 01:00:49.393881 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:00:49.394362 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:00:49.395988 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:00:49.396788 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:00:49.397236 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:00:49.399822 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:00:49.402012 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO dbcounter [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Registered counter for database nova_cell0 Aug 07 01:00:49.408381 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO dbcounter [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Registered counter for database nova_cell1 Aug 07 01:00:49.419478 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_db.sqlalchemy.engines [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103442) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:00:49.423692 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_db.sqlalchemy.engines [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103442) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:00:49.433374 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writer thread running {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:00:49.443725 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writer thread running {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:00:49.555229 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO dbcounter [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] Registered counter for database nova_api Aug 07 01:00:49.590552 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG oslo_db.sqlalchemy.engines [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103441) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:00:49.602419 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG dbcounter [-] [103441] Writer thread running {{(pid=103441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:00:49.710091 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.context [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dfea601e-2ab8-4cc9-8a98-b642673a3d20(cell1) {{(pid=103441) load_cells /opt/stack/nova/nova/context.py:464}} Aug 07 01:00:49.717545 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG oslo_concurrency.lockutils [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103441) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:00:49.718392 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG oslo_concurrency.lockutils [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103441) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:00:49.721721 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG oslo_concurrency.lockutils [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=103441) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:00:49.723001 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG oslo_concurrency.lockutils [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103441) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:00:49.723789 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG oslo_concurrency.lockutils [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103441) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:00:49.727156 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO dbcounter [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Registered counter for database nova_cell0 Aug 07 01:00:49.728163 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG oslo_concurrency.lockutils [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=103441) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:00:49.734307 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO dbcounter [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] Registered counter for database nova_cell0 Aug 07 01:00:49.744557 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO dbcounter [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] Registered counter for database nova_cell1 Aug 07 01:00:49.746353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_db.sqlalchemy.engines [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103442) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:00:49.753099 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writer thread running {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:00:49.756555 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG oslo_db.sqlalchemy.engines [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103441) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:00:49.762048 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG oslo_db.sqlalchemy.engines [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103441) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:00:49.767706 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG dbcounter [-] [103441] Writer thread running {{(pid=103441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:00:50.025623 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG dbcounter [-] [103441] Writer thread running {{(pid=103441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:00:50.044407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO dbcounter [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] Registered counter for database nova_cell0 Aug 07 01:00:50.059932 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG oslo_db.sqlalchemy.engines [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103441) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:00:50.067752 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG dbcounter [-] [103441] Writer thread running {{(pid=103441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:00:50.115451 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: WARNING keystonemiddleware.auth_token [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 07 01:00:50.123678 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: WARNING keystonemiddleware.auth_token [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 07 01:00:50.486447 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: WARNING keystonemiddleware.auth_token [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 07 01:00:50.493056 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: WARNING keystonemiddleware.auth_token [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 07 01:00:50.498415 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5572c17369c0 pid: 103442 (default app) Aug 07 01:00:50.500651 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:00:50.501852 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] 10.222.0.26 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001898 Aug 07 01:00:50.502262 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 788 bytes} [Sun Aug 7 01:00:50 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:00:50.556729 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: WARNING keystonemiddleware.auth_token [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 07 01:00:50.565752 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: WARNING keystonemiddleware.auth_token [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 07 01:00:51.082021 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: WARNING keystonemiddleware.auth_token [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 07 01:00:51.094767 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: WARNING keystonemiddleware.auth_token [None req-745d93d5-b953-408a-9bb4-a8e28880b9cc None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 07 01:00:51.106738 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5572c17369c0 pid: 103441 (default app) Aug 07 01:02:30.479346 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:02:30.481233 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002327 Aug 07 01:02:30.481842 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 2/2] 10.222.0.26 () {54 vars in 931 bytes} [Sun Aug 7 01:02:30 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:02:30.498650 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG dbcounter [-] [103441] Writing DB stats nova_api:SELECT=3 {{(pid=103441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:02:30.500376 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG dbcounter [-] [103441] Writing DB stats nova_cell0:SELECT=3 {{(pid=103441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:02:30.502262 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG dbcounter [-] [103441] Writing DB stats nova_cell1:SELECT=3 {{(pid=103441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:02:30.504065 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG dbcounter [-] [103441] Writing DB stats nova_cell0:SELECT=3 {{(pid=103441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:02:31.661106 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-005932ce-1d85-4ab4-8e34-d5f3ff26846f admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:02:31.701059 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-005932ce-1d85-4ab4-8e34-d5f3ff26846f admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 1.209008 Aug 07 01:02:31.716277 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 1/3] 10.222.0.26 () {60 vars in 1292 bytes} [Sun Aug 7 01:02:30 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 1212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:02:34.530793 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:02:34.533053 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-7c47595b-5fef-474a-9789-fe289a42edb8 None None] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002580 Aug 07 01:02:34.533765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 3/4] 10.222.0.26 () {54 vars in 931 bytes} [Sun Aug 7 01:02:34 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:02:35.149996 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-291609db-649f-4ab6-9d97-09be0e081a5f admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "ram": 256, "disk": 1, "name": "cirros256", "id": "c1", "swap": 0, "vcpus": 1}} {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:02:35.172629 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 01:02:35.172629 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: warnings.warn(msg) Aug 07 01:02:35.207976 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-291609db-649f-4ab6-9d97-09be0e081a5f admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.666165 Aug 07 01:02:35.208962 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 2/5] 10.222.0.26 () {64 vars in 1283 bytes} [Sun Aug 7 01:02:34 2022] POST /compute/v2.1/flavors => generated 407 bytes in 668 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 07 01:02:35.438696 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-041e378e-8389-48ed-9278-f11b57447e6f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:02:35.443234 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 01:02:35.443234 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: warnings.warn(msg) Aug 07 01:02:35.446947 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writing DB stats nova_api:SELECT=3 {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:02:35.448471 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writing DB stats nova_cell0:SELECT=3 {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:02:35.449956 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writing DB stats nova_cell1:SELECT=3 {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:02:35.450852 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:02:35.526683 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-041e378e-8389-48ed-9278-f11b57447e6f admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.308245 Aug 07 01:02:35.532490 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 4/6] 10.222.0.26 () {64 vars in 1336 bytes} [Sun Aug 7 01:02:35 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:02:38.165341 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-291609db-649f-4ab6-9d97-09be0e081a5f admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:02:38.169889 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-291609db-649f-4ab6-9d97-09be0e081a5f admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003500 Aug 07 01:02:38.171104 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 3/7] 10.222.0.26 () {54 vars in 931 bytes} [Sun Aug 7 01:02:38 2022] GET /compute/v2.1 => generated 388 bytes in 7 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:02:39.224730 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-61018dba-bb2d-49e4-a55e-898325cb9536 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "disk": 5, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 512, "rxtx_factor": 1.0, "id": "d1", "swap": 0, "name": "ds512M"}} {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:02:39.248304 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 01:02:39.248304 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: warnings.warn(msg) Aug 07 01:02:39.276957 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-61018dba-bb2d-49e4-a55e-898325cb9536 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 1.094455 Aug 07 01:02:39.278208 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 5/8] 10.222.0.26 () {64 vars in 1283 bytes} [Sun Aug 7 01:02:38 2022] POST /compute/v2.1/flavors => generated 404 bytes in 1097 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:02:39.301730 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-245bbf1c-e2a9-44fc-9471-5c4a55b0cc24 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:02:39.306021 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 01:02:39.306021 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: warnings.warn(msg) Aug 07 01:02:39.357461 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-245bbf1c-e2a9-44fc-9471-5c4a55b0cc24 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.064558 Aug 07 01:02:39.358339 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 4/9] 10.222.0.26 () {64 vars in 1336 bytes} [Sun Aug 7 01:02:39 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:02:42.520333 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-61018dba-bb2d-49e4-a55e-898325cb9536 admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:02:42.521431 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-61018dba-bb2d-49e4-a55e-898325cb9536 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001551 Aug 07 01:02:42.521858 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 6/10] 10.222.0.26 () {54 vars in 931 bytes} [Sun Aug 7 01:02:42 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:02:43.231275 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-36ca23b5-be89-4309-badf-35ccd722b75c admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "disk": 10, "ram": 1024, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "id": "d2", "name": "ds1G", "swap": 0}} {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:02:43.270684 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-36ca23b5-be89-4309-badf-35ccd722b75c admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.741009 Aug 07 01:02:43.271729 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 5/11] 10.222.0.26 () {64 vars in 1283 bytes} [Sun Aug 7 01:02:42 2022] POST /compute/v2.1/flavors => generated 404 bytes in 743 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 07 01:02:43.294044 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-d0045b89-728b-475e-ba81-fb76aa08145d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:02:43.334982 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-d0045b89-728b-475e-ba81-fb76aa08145d admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.047160 Aug 07 01:02:43.336411 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 7/12] 10.222.0.26 () {64 vars in 1336 bytes} [Sun Aug 7 01:02:43 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:02:46.091726 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-36ca23b5-be89-4309-badf-35ccd722b75c admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:02:46.093628 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-36ca23b5-be89-4309-badf-35ccd722b75c admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002509 Aug 07 01:02:46.094755 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 6/13] 10.222.0.26 () {54 vars in 931 bytes} [Sun Aug 7 01:02:46 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:02:46.749706 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-6035c71a-b318-48eb-b9e4-08b3d79e6f85 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds2G", "vcpus": 2, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d3", "ram": 2048, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "disk": 10, "swap": 0}} {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:02:46.774420 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-6035c71a-b318-48eb-b9e4-08b3d79e6f85 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.672008 Aug 07 01:02:46.775642 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 8/14] 10.222.0.26 () {64 vars in 1283 bytes} [Sun Aug 7 01:02:46 2022] POST /compute/v2.1/flavors => generated 404 bytes in 674 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:02:46.799619 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-e150fdf3-b0bd-43a8-9685-d07ed6ab869f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:02:46.841419 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-e150fdf3-b0bd-43a8-9685-d07ed6ab869f admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.049133 Aug 07 01:02:46.852524 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 7/15] 10.222.0.26 () {64 vars in 1336 bytes} [Sun Aug 7 01:02:46 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:02:50.137614 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-6035c71a-b318-48eb-b9e4-08b3d79e6f85 admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:02:50.139598 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-6035c71a-b318-48eb-b9e4-08b3d79e6f85 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002384 Aug 07 01:02:50.139961 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 9/16] 10.222.0.26 () {54 vars in 931 bytes} [Sun Aug 7 01:02:50 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:02:50.677934 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-ea549e8c-a835-47b4-af83-fdd2da7a71d1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "swap": 0, "os-flavor-access:is_public": true, "disk": 20, "name": "ds4G", "ram": 4096, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 4, "id": "d4"}} {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:02:50.715017 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-ea549e8c-a835-47b4-af83-fdd2da7a71d1 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.567111 Aug 07 01:02:50.720021 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 8/17] 10.222.0.26 () {64 vars in 1283 bytes} [Sun Aug 7 01:02:50 2022] POST /compute/v2.1/flavors => generated 404 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:02:50.735851 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-56b21e46-8bc9-44d4-9a23-79c88a029e2a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:02:50.780720 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-56b21e46-8bc9-44d4-9a23-79c88a029e2a admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.050440 Aug 07 01:02:50.782450 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 10/18] 10.222.0.26 () {64 vars in 1336 bytes} [Sun Aug 7 01:02:50 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:02:54.408553 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-ea549e8c-a835-47b4-af83-fdd2da7a71d1 admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:02:54.410013 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-ea549e8c-a835-47b4-af83-fdd2da7a71d1 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002460 Aug 07 01:02:54.410586 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 9/19] 10.222.0.26 () {54 vars in 931 bytes} [Sun Aug 7 01:02:54 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:02:55.139730 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-9b9369b6-f23c-47b1-bea7-f870702ea7c5 admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:02:55.178260 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-9b9369b6-f23c-47b1-bea7-f870702ea7c5 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.758808 Aug 07 01:02:55.179296 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 11/20] 10.222.0.26 () {60 vars in 1292 bytes} [Sun Aug 7 01:02:54 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 761 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:02:58.063010 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-ea549e8c-a835-47b4-af83-fdd2da7a71d1 admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:02:58.064880 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-ea549e8c-a835-47b4-af83-fdd2da7a71d1 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002405 Aug 07 01:02:58.065631 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 10/21] 10.222.0.26 () {54 vars in 931 bytes} [Sun Aug 7 01:02:58 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:02:58.706812 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-40673e5e-23b3-4a0c-b2ac-22711d2685e2 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.tiny", "disk": 1, "os-flavor-access:is_public": true, "vcpus": 1, "swap": 0, "id": "1", "ram": 512, "rxtx_factor": 1.0}} {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:02:58.733551 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-40673e5e-23b3-4a0c-b2ac-22711d2685e2 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.655277 Aug 07 01:02:58.734806 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 12/22] 10.222.0.26 () {64 vars in 1283 bytes} [Sun Aug 7 01:02:58 2022] POST /compute/v2.1/flavors => generated 402 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 07 01:02:58.755650 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-ee911aa1-683a-4f81-a83c-2e638167ff15 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:02:58.805843 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-ee911aa1-683a-4f81-a83c-2e638167ff15 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.058780 Aug 07 01:02:58.806441 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 11/23] 10.222.0.26 () {64 vars in 1333 bytes} [Sun Aug 7 01:02:58 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:03:02.247634 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-40673e5e-23b3-4a0c-b2ac-22711d2685e2 admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:02.248934 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-40673e5e-23b3-4a0c-b2ac-22711d2685e2 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001751 Aug 07 01:03:02.249536 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 13/24] 10.222.0.26 () {54 vars in 931 bytes} [Sun Aug 7 01:03:02 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:03:02.890788 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-c6dd78ea-df83-4284-a24b-684fe46f153b admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "ram": 2048, "os-flavor-access:is_public": true, "name": "m1.small", "OS-FLV-EXT-DATA:ephemeral": 0, "id": "2", "swap": 0, "vcpus": 1, "disk": 20}} {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:03:02.896832 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG dbcounter [-] [103441] Writing DB stats nova_api:SELECT=30,nova_api:INSERT=6,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=103441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:03:02.928256 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-c6dd78ea-df83-4284-a24b-684fe46f153b admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.670639 Aug 07 01:03:02.929132 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 12/25] 10.222.0.26 () {64 vars in 1283 bytes} [Sun Aug 7 01:03:02 2022] POST /compute/v2.1/flavors => generated 405 bytes in 673 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 07 01:03:02.948088 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-dfdd9284-02f1-47a1-9baa-786b7513548c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:03:02.982844 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-dfdd9284-02f1-47a1-9baa-786b7513548c admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.038565 Aug 07 01:03:02.983818 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 14/26] 10.222.0.26 () {64 vars in 1333 bytes} [Sun Aug 7 01:03:02 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:03:06.209625 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-c6dd78ea-df83-4284-a24b-684fe46f153b admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:06.211000 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-c6dd78ea-df83-4284-a24b-684fe46f153b admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001979 Aug 07 01:03:06.211564 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 13/27] 10.222.0.26 () {54 vars in 931 bytes} [Sun Aug 7 01:03:06 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:03:06.895581 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-8f36db45-a05a-4ace-9f20-ba9d44e597f9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "vcpus": 2, "id": "3", "name": "m1.medium", "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 4096, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "disk": 40}} {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:03:06.908896 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writing DB stats nova_api:SELECT=36,nova_api:SAVEPOINT=4,nova_api:INSERT=7,nova_api:RELEASE=4 {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:03:06.936062 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-8f36db45-a05a-4ace-9f20-ba9d44e597f9 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.713632 Aug 07 01:03:06.937638 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 15/28] 10.222.0.26 () {64 vars in 1283 bytes} [Sun Aug 7 01:03:06 2022] POST /compute/v2.1/flavors => generated 406 bytes in 716 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:03:06.963600 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-d67bfac5-4718-47c2-8997-6ac5ba92e8e6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:03:07.004652 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-d67bfac5-4718-47c2-8997-6ac5ba92e8e6 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.050759 Aug 07 01:03:07.006322 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 14/29] 10.222.0.26 () {64 vars in 1333 bytes} [Sun Aug 7 01:03:06 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:03:10.110509 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-8f36db45-a05a-4ace-9f20-ba9d44e597f9 admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:10.112883 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-8f36db45-a05a-4ace-9f20-ba9d44e597f9 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003913 Aug 07 01:03:10.113405 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 16/30] 10.222.0.26 () {54 vars in 931 bytes} [Sun Aug 7 01:03:10 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:03:10.888525 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-0a4dd767-6870-43fb-b280-1562339a661a admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 4, "id": "4", "ram": 8192, "name": "m1.large", "os-flavor-access:is_public": true, "disk": 80, "rxtx_factor": 1.0, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:03:10.922956 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-0a4dd767-6870-43fb-b280-1562339a661a admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.799735 Aug 07 01:03:10.933375 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 15/31] 10.222.0.26 () {64 vars in 1283 bytes} [Sun Aug 7 01:03:10 2022] POST /compute/v2.1/flavors => generated 405 bytes in 811 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 07 01:03:10.940752 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-f89bd92c-51dd-4161-9e92-a4661ac3eb31 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:03:10.981320 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-f89bd92c-51dd-4161-9e92-a4661ac3eb31 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.045846 Aug 07 01:03:10.982103 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 17/32] 10.222.0.26 () {64 vars in 1333 bytes} [Sun Aug 7 01:03:10 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:03:13.922750 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-0a4dd767-6870-43fb-b280-1562339a661a admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:13.925925 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-0a4dd767-6870-43fb-b280-1562339a661a admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.004438 Aug 07 01:03:13.926695 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 16/33] 10.222.0.26 () {54 vars in 931 bytes} [Sun Aug 7 01:03:13 2022] GET /compute/v2.1 => generated 388 bytes in 7 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:03:14.636701 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-35d5a9bf-f44e-45cb-b58d-c52c2accb12e admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 160, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "os-flavor-access:is_public": true, "vcpus": 8, "rxtx_factor": 1.0, "ram": 16384, "name": "m1.xlarge", "id": "5"}} {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:03:14.661748 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-35d5a9bf-f44e-45cb-b58d-c52c2accb12e admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.725649 Aug 07 01:03:14.662393 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 18/34] 10.222.0.26 () {64 vars in 1283 bytes} [Sun Aug 7 01:03:13 2022] POST /compute/v2.1/flavors => generated 408 bytes in 727 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:03:14.677058 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-35ec7b37-61af-4fe0-a2e5-af2f56a945d4 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:03:14.715407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-35ec7b37-61af-4fe0-a2e5-af2f56a945d4 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.044337 Aug 07 01:03:14.715939 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 17/35] 10.222.0.26 () {64 vars in 1333 bytes} [Sun Aug 7 01:03:14 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:03:35.101442 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-35d5a9bf-f44e-45cb-b58d-c52c2accb12e admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:35.102384 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-35d5a9bf-f44e-45cb-b58d-c52c2accb12e admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001473 Aug 07 01:03:35.102793 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 19/36] 10.222.0.26 () {54 vars in 931 bytes} [Sun Aug 7 01:03:35 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:03:35.115127 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG dbcounter [-] [103441] Writing DB stats nova_api:INSERT=4,nova_api:SELECT=17,nova_api:SAVEPOINT=2,nova_api:RELEASE=2 {{(pid=103441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:03:35.695528 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-64ff06a8-5671-4df2-a10c-0bae4e11ec15 admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:35.715019 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-64ff06a8-5671-4df2-a10c-0bae4e11ec15 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.605378 Aug 07 01:03:35.715593 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 18/37] 10.222.0.26 () {60 vars in 1292 bytes} [Sun Aug 7 01:03:35 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 607 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:03:37.145093 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-35d5a9bf-f44e-45cb-b58d-c52c2accb12e admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:37.147033 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-35d5a9bf-f44e-45cb-b58d-c52c2accb12e admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002608 Aug 07 01:03:37.147820 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 20/38] 10.222.0.26 () {56 vars in 1074 bytes} [Sun Aug 7 01:03:37 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:03:37.849327 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-e6dc96c0-30d2-480b-94bb-a3b65e986f3d admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:37.855900 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.wsgi [None req-e6dc96c0-30d2-480b-94bb-a3b65e986f3d admin admin] HTTP exception thrown: Flavor 42 could not be found. Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-e6dc96c0-30d2-480b-94bb-a3b65e986f3d admin admin] Request method failure captured: Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Accept: application/json Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Accept-Encoding: gzip, deflate Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Connection: keep-alive Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Content-Length: 0 Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Host: 10.222.0.26 Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Openstack-Api-Version: compute 2.90 Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Openstack-System-Scope: None Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: User-Agent: python-novaclient Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Auth-Token: gAAAAABi7w9oUbiEla-9pW6goJB83IYoWCn0-9lqIGlulGEwJ_ieKFJMQzkLSKxMphLHmblmdocc_To7ST555uit6ayP-GPNaqxLVjr8iIIg1tYUQF3WBfJoxGq-p3cHGBENjOBdF5gqv_df-Trh1ADFJXpRrrpuzr84Gac6rq5YcSyj4Jm8rC0 Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Domain-Id: None Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Domain-Name: None Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Identity-Status: Confirmed Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Is-Admin-Project: True Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Openstack-Nova-Api-Version: 2.90 Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Project-Domain-Id: default Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Project-Domain-Name: Default Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Project-Id: ae3747bdb31f45b9bc97b22fb3bfe021 Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Project-Name: admin Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Role: member,reader,admin Aug 07 01:03:37.859407 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Roles: member,reader,admin Aug 07 01:03:37.861632 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:03:37.861632 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Tenant: admin Aug 07 01:03:37.861632 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Tenant-Id: ae3747bdb31f45b9bc97b22fb3bfe021 Aug 07 01:03:37.861632 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Tenant-Name: admin Aug 07 01:03:37.861632 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User: admin Aug 07 01:03:37.861632 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User-Domain-Id: default Aug 07 01:03:37.861632 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User-Domain-Name: Default Aug 07 01:03:37.861632 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User-Id: a75da33429f640299cbc9644c9a49291 Aug 07 01:03:37.861632 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User-Name: admin Aug 07 01:03:37.861632 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: method: > Aug 07 01:03:37.861632 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: exception: Flavor 42 could not be found. Aug 07 01:03:37.861632 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: action_args: {'id': '42'} Aug 07 01:03:37.861632 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:03:37.861632 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:03:37.862992 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:37.866334 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-e6dc96c0-30d2-480b-94bb-a3b65e986f3d admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103441) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:03:37.869245 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-e6dc96c0-30d2-480b-94bb-a3b65e986f3d admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.551579 Aug 07 01:03:37.869245 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 19/39] 10.222.0.26 () {60 vars in 1190 bytes} [Sun Aug 7 01:03:37 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 553 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 07 01:03:37.876637 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-d8265e1f-15db-415b-b56e-23a68a74c21a admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:37.905139 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writing DB stats nova_api:INSERT=3,nova_api:SELECT=11,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:03:37.914924 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-d8265e1f-15db-415b-b56e-23a68a74c21a admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.045392 Aug 07 01:03:37.924322 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 21/40] 10.222.0.26 () {60 vars in 1210 bytes} [Sun Aug 7 01:03:37 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:03:37.930208 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-84d906fb-7fad-4854-8994-b8b9c0024654 admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:37.963442 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-84d906fb-7fad-4854-8994-b8b9c0024654 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.037156 Aug 07 01:03:37.964282 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 20/41] 10.222.0.26 () {60 vars in 1210 bytes} [Sun Aug 7 01:03:37 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:03:37.975918 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-99b6d1a8-6deb-4ea7-992d-66b3c289304f admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:37.988610 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.wsgi [None req-99b6d1a8-6deb-4ea7-992d-66b3c289304f admin admin] HTTP exception thrown: Flavor 42 could not be found. Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-99b6d1a8-6deb-4ea7-992d-66b3c289304f admin admin] Request method failure captured: Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Accept: application/json Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Accept-Encoding: gzip, deflate Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Connection: keep-alive Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Content-Length: 0 Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Host: 10.222.0.26 Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Openstack-Api-Version: compute 2.90 Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Openstack-System-Scope: None Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: User-Agent: python-novaclient Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Auth-Token: gAAAAABi7w9oUbiEla-9pW6goJB83IYoWCn0-9lqIGlulGEwJ_ieKFJMQzkLSKxMphLHmblmdocc_To7ST555uit6ayP-GPNaqxLVjr8iIIg1tYUQF3WBfJoxGq-p3cHGBENjOBdF5gqv_df-Trh1ADFJXpRrrpuzr84Gac6rq5YcSyj4Jm8rC0 Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Domain-Id: None Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Domain-Name: None Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Identity-Status: Confirmed Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Is-Admin-Project: True Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Openstack-Nova-Api-Version: 2.90 Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Project-Domain-Id: default Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Project-Domain-Name: Default Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Project-Id: ae3747bdb31f45b9bc97b22fb3bfe021 Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Project-Name: admin Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Role: member,reader,admin Aug 07 01:03:37.992693 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Roles: member,reader,admin Aug 07 01:03:37.994353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:03:37.994353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Tenant: admin Aug 07 01:03:37.994353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Tenant-Id: ae3747bdb31f45b9bc97b22fb3bfe021 Aug 07 01:03:37.994353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Tenant-Name: admin Aug 07 01:03:37.994353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User: admin Aug 07 01:03:37.994353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User-Domain-Id: default Aug 07 01:03:37.994353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User-Domain-Name: Default Aug 07 01:03:37.994353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User-Id: a75da33429f640299cbc9644c9a49291 Aug 07 01:03:37.994353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User-Name: admin Aug 07 01:03:37.994353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: method: > Aug 07 01:03:37.994353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: exception: Flavor 42 could not be found. Aug 07 01:03:37.994353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: action_args: {'id': '42'} Aug 07 01:03:37.994353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:03:37.994353 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:03:37.995897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:37.997765 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-99b6d1a8-6deb-4ea7-992d-66b3c289304f admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103442) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:03:37.999947 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-99b6d1a8-6deb-4ea7-992d-66b3c289304f admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.022355 Aug 07 01:03:37.999947 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 22/42] 10.222.0.26 () {60 vars in 1190 bytes} [Sun Aug 7 01:03:37 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 24 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 07 01:03:39.119209 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-84d906fb-7fad-4854-8994-b8b9c0024654 admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:39.120251 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-84d906fb-7fad-4854-8994-b8b9c0024654 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001473 Aug 07 01:03:39.120683 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 21/43] 10.222.0.26 () {56 vars in 1074 bytes} [Sun Aug 7 01:03:39 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:03:39.815832 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-dd1b6edc-839b-449f-8329-efae6e90880c admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:39.822862 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.wsgi [None req-dd1b6edc-839b-449f-8329-efae6e90880c admin admin] HTTP exception thrown: Flavor 84 could not be found. Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-dd1b6edc-839b-449f-8329-efae6e90880c admin admin] Request method failure captured: Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Accept: application/json Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Accept-Encoding: gzip, deflate Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Connection: keep-alive Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Content-Length: 0 Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Host: 10.222.0.26 Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Openstack-Api-Version: compute 2.90 Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Openstack-System-Scope: None Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: User-Agent: python-novaclient Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Auth-Token: gAAAAABi7w9qcUbxG8HrKz7JlXqXY0t-74seqkdLE3JzJ5Mc_PTE2IxLzE7x-6BqiGnjQJ1gO7AX0j1GpNLoo4wsbHyheZp2oMLJPlGBMyJLYAwBjGqbIxDVMID1yyW-6TqXIZ6S0WDMFku5bMM81l_LjZfXgkTHYIffnUgcT3ciRsJJ1KFcgLk Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Domain-Id: None Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Domain-Name: None Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Identity-Status: Confirmed Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Is-Admin-Project: True Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Openstack-Nova-Api-Version: 2.90 Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Project-Domain-Id: default Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Project-Domain-Name: Default Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Project-Id: ae3747bdb31f45b9bc97b22fb3bfe021 Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Project-Name: admin Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Role: member,reader,admin Aug 07 01:03:39.824446 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Roles: member,reader,admin Aug 07 01:03:39.825973 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:03:39.825973 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Tenant: admin Aug 07 01:03:39.825973 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Tenant-Id: ae3747bdb31f45b9bc97b22fb3bfe021 Aug 07 01:03:39.825973 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Tenant-Name: admin Aug 07 01:03:39.825973 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User: admin Aug 07 01:03:39.825973 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User-Domain-Id: default Aug 07 01:03:39.825973 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User-Domain-Name: Default Aug 07 01:03:39.825973 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User-Id: a75da33429f640299cbc9644c9a49291 Aug 07 01:03:39.825973 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User-Name: admin Aug 07 01:03:39.825973 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: method: > Aug 07 01:03:39.825973 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: exception: Flavor 84 could not be found. Aug 07 01:03:39.825973 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: action_args: {'id': '84'} Aug 07 01:03:39.825973 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:03:39.825973 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:03:39.827007 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:39.828297 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-dd1b6edc-839b-449f-8329-efae6e90880c admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103442) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:03:39.829434 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-dd1b6edc-839b-449f-8329-efae6e90880c admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.560295 Aug 07 01:03:39.829434 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 23/44] 10.222.0.26 () {60 vars in 1190 bytes} [Sun Aug 7 01:03:39 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 563 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 07 01:03:39.840083 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-988226b7-c269-4ec0-98ab-6fd82b301e22 admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:39.856527 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-988226b7-c269-4ec0-98ab-6fd82b301e22 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.022748 Aug 07 01:03:39.857098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 22/45] 10.222.0.26 () {60 vars in 1210 bytes} [Sun Aug 7 01:03:39 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 24 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:03:39.867858 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-9d12da4b-07ed-4cae-bbe0-87a4d29facc9 admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:39.885567 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-9d12da4b-07ed-4cae-bbe0-87a4d29facc9 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.021244 Aug 07 01:03:39.886137 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 24/46] 10.222.0.26 () {60 vars in 1210 bytes} [Sun Aug 7 01:03:39 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 23 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:03:39.897055 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-e4dc43d3-94b2-48d3-9bb3-ed3a08a148c2 admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:39.902949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.wsgi [None req-e4dc43d3-94b2-48d3-9bb3-ed3a08a148c2 admin admin] HTTP exception thrown: Flavor 84 could not be found. Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-e4dc43d3-94b2-48d3-9bb3-ed3a08a148c2 admin admin] Request method failure captured: Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Accept: application/json Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Accept-Encoding: gzip, deflate Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Connection: keep-alive Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Content-Length: 0 Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Host: 10.222.0.26 Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Openstack-Api-Version: compute 2.90 Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Openstack-System-Scope: None Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: User-Agent: python-novaclient Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Auth-Token: gAAAAABi7w9qcUbxG8HrKz7JlXqXY0t-74seqkdLE3JzJ5Mc_PTE2IxLzE7x-6BqiGnjQJ1gO7AX0j1GpNLoo4wsbHyheZp2oMLJPlGBMyJLYAwBjGqbIxDVMID1yyW-6TqXIZ6S0WDMFku5bMM81l_LjZfXgkTHYIffnUgcT3ciRsJJ1KFcgLk Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Domain-Id: None Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Domain-Name: None Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Identity-Status: Confirmed Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Is-Admin-Project: True Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Openstack-Nova-Api-Version: 2.90 Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Project-Domain-Id: default Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Project-Domain-Name: Default Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Project-Id: ae3747bdb31f45b9bc97b22fb3bfe021 Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Project-Name: admin Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Role: member,reader,admin Aug 07 01:03:39.905098 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Roles: member,reader,admin Aug 07 01:03:39.907588 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:03:39.907588 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Tenant: admin Aug 07 01:03:39.907588 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Tenant-Id: ae3747bdb31f45b9bc97b22fb3bfe021 Aug 07 01:03:39.907588 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Tenant-Name: admin Aug 07 01:03:39.907588 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User: admin Aug 07 01:03:39.907588 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User-Domain-Id: default Aug 07 01:03:39.907588 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User-Domain-Name: Default Aug 07 01:03:39.907588 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User-Id: a75da33429f640299cbc9644c9a49291 Aug 07 01:03:39.907588 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User-Name: admin Aug 07 01:03:39.907588 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: method: > Aug 07 01:03:39.907588 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: exception: Flavor 84 could not be found. Aug 07 01:03:39.907588 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: action_args: {'id': '84'} Aug 07 01:03:39.907588 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:03:39.907588 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:03:39.909454 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:39.911482 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-e4dc43d3-94b2-48d3-9bb3-ed3a08a148c2 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103441) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:03:39.913661 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-e4dc43d3-94b2-48d3-9bb3-ed3a08a148c2 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.013297 Aug 07 01:03:39.914207 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 23/47] 10.222.0.26 () {60 vars in 1190 bytes} [Sun Aug 7 01:03:39 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 22 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 07 01:03:40.986161 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-9d12da4b-07ed-4cae-bbe0-87a4d29facc9 admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:40.987252 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-9d12da4b-07ed-4cae-bbe0-87a4d29facc9 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001554 Aug 07 01:03:40.987749 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 25/48] 10.222.0.26 () {56 vars in 1074 bytes} [Sun Aug 7 01:03:40 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:03:41.580115 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-8f80e5f6-ae41-4681-92be-5046f8f51a35 admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:41.586411 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.wsgi [None req-8f80e5f6-ae41-4681-92be-5046f8f51a35 admin admin] HTTP exception thrown: Flavor 451 could not be found. Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-8f80e5f6-ae41-4681-92be-5046f8f51a35 admin admin] Request method failure captured: Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Accept: application/json Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Accept-Encoding: gzip, deflate Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Connection: keep-alive Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Content-Length: 0 Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Host: 10.222.0.26 Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Openstack-Api-Version: compute 2.90 Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Openstack-System-Scope: None Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: User-Agent: python-novaclient Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Auth-Token: gAAAAABi7w9svCNJh7QJ9-c5P1RA2DFD7HHLjbmraym76O43zpmsKmOlc9wVeVZGrpzO1yOvd69f8qbtLYMTAEPn5sxXz_tnkY9q7VS76iBGX9shnv3uqEwjJ-UW9eknp9E7MFIaoQrZ4LJzHbUzk-xuj8HubLr4LLBdqHv_YpTu2GikAepYjQY Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Domain-Id: None Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Domain-Name: None Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Identity-Status: Confirmed Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Is-Admin-Project: True Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Openstack-Nova-Api-Version: 2.90 Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Project-Domain-Id: default Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Project-Domain-Name: Default Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Project-Id: ae3747bdb31f45b9bc97b22fb3bfe021 Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Project-Name: admin Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Role: member,reader,admin Aug 07 01:03:41.587949 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Roles: member,reader,admin Aug 07 01:03:41.589336 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:03:41.589336 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Tenant: admin Aug 07 01:03:41.589336 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Tenant-Id: ae3747bdb31f45b9bc97b22fb3bfe021 Aug 07 01:03:41.589336 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-Tenant-Name: admin Aug 07 01:03:41.589336 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User: admin Aug 07 01:03:41.589336 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User-Domain-Id: default Aug 07 01:03:41.589336 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User-Domain-Name: Default Aug 07 01:03:41.589336 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User-Id: a75da33429f640299cbc9644c9a49291 Aug 07 01:03:41.589336 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: X-User-Name: admin Aug 07 01:03:41.589336 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: method: > Aug 07 01:03:41.589336 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: exception: Flavor 451 could not be found. Aug 07 01:03:41.589336 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: action_args: {'id': '451'} Aug 07 01:03:41.589336 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:03:41.589336 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:03:41.590148 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: ERROR nova.api.openstack.wsgi Aug 07 01:03:41.591230 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-8f80e5f6-ae41-4681-92be-5046f8f51a35 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103441) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:03:41.593222 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-8f80e5f6-ae41-4681-92be-5046f8f51a35 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.455883 Aug 07 01:03:41.593222 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 24/49] 10.222.0.26 () {60 vars in 1193 bytes} [Sun Aug 7 01:03:41 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 457 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 07 01:03:41.604285 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-332aaf11-3684-405c-ad5a-7952c26627bb admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:41.620023 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-332aaf11-3684-405c-ad5a-7952c26627bb admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.020373 Aug 07 01:03:41.620683 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 26/50] 10.222.0.26 () {60 vars in 1210 bytes} [Sun Aug 7 01:03:41 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:03:41.631374 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-4b5af8bf-8fbc-48a5-b55d-bd82519b0e9d admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:41.655106 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-4b5af8bf-8fbc-48a5-b55d-bd82519b0e9d admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.026599 Aug 07 01:03:41.655739 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 25/51] 10.222.0.26 () {60 vars in 1210 bytes} [Sun Aug 7 01:03:41 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:03:41.666405 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-3032b7d3-327e-4eb3-9bd4-c24a29e23b2d admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:41.672425 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.wsgi [None req-3032b7d3-327e-4eb3-9bd4-c24a29e23b2d admin admin] HTTP exception thrown: Flavor 451 could not be found. Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-3032b7d3-327e-4eb3-9bd4-c24a29e23b2d admin admin] Request method failure captured: Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Accept: application/json Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Accept-Encoding: gzip, deflate Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Connection: keep-alive Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Content-Length: 0 Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Host: 10.222.0.26 Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Openstack-Api-Version: compute 2.90 Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Openstack-System-Scope: None Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: User-Agent: python-novaclient Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Auth-Token: gAAAAABi7w9svCNJh7QJ9-c5P1RA2DFD7HHLjbmraym76O43zpmsKmOlc9wVeVZGrpzO1yOvd69f8qbtLYMTAEPn5sxXz_tnkY9q7VS76iBGX9shnv3uqEwjJ-UW9eknp9E7MFIaoQrZ4LJzHbUzk-xuj8HubLr4LLBdqHv_YpTu2GikAepYjQY Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Domain-Id: None Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Domain-Name: None Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Identity-Status: Confirmed Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Is-Admin-Project: True Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Openstack-Nova-Api-Version: 2.90 Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Project-Domain-Id: default Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Project-Domain-Name: Default Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Project-Id: ae3747bdb31f45b9bc97b22fb3bfe021 Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Project-Name: admin Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Role: member,reader,admin Aug 07 01:03:41.673969 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Roles: member,reader,admin Aug 07 01:03:41.677060 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ae3747bdb31f45b9bc97b22fb3bfe021"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:03:41.677060 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Tenant: admin Aug 07 01:03:41.677060 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Tenant-Id: ae3747bdb31f45b9bc97b22fb3bfe021 Aug 07 01:03:41.677060 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-Tenant-Name: admin Aug 07 01:03:41.677060 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User: admin Aug 07 01:03:41.677060 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User-Domain-Id: default Aug 07 01:03:41.677060 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User-Domain-Name: Default Aug 07 01:03:41.677060 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User-Id: a75da33429f640299cbc9644c9a49291 Aug 07 01:03:41.677060 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: X-User-Name: admin Aug 07 01:03:41.677060 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: method: > Aug 07 01:03:41.677060 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: exception: Flavor 451 could not be found. Aug 07 01:03:41.677060 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: action_args: {'id': '451'} Aug 07 01:03:41.677060 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:03:41.677060 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:03:41.678003 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: ERROR nova.api.openstack.wsgi Aug 07 01:03:41.679247 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-3032b7d3-327e-4eb3-9bd4-c24a29e23b2d admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103442) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:03:41.680534 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-3032b7d3-327e-4eb3-9bd4-c24a29e23b2d admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.012874 Aug 07 01:03:41.680534 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 27/52] 10.222.0.26 () {60 vars in 1193 bytes} [Sun Aug 7 01:03:41 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 14 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 07 01:03:42.752523 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-4b5af8bf-8fbc-48a5-b55d-bd82519b0e9d admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:42.753593 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-4b5af8bf-8fbc-48a5-b55d-bd82519b0e9d admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001997 Aug 07 01:03:42.754007 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 26/53] 10.222.0.26 () {56 vars in 1074 bytes} [Sun Aug 7 01:03:42 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:03:43.404206 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-bf8007c4-823b-47bf-a392-bef29bca0afc admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:03:43.426258 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-bf8007c4-823b-47bf-a392-bef29bca0afc admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.524624 Aug 07 01:03:43.426818 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 28/54] 10.222.0.26 () {64 vars in 1235 bytes} [Sun Aug 7 01:03:42 2022] POST /compute/v2.1/flavors => generated 404 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:03:44.736055 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-4b5af8bf-8fbc-48a5-b55d-bd82519b0e9d admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:44.737103 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-4b5af8bf-8fbc-48a5-b55d-bd82519b0e9d admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001492 Aug 07 01:03:44.737506 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 27/55] 10.222.0.26 () {56 vars in 1074 bytes} [Sun Aug 7 01:03:44 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:03:45.290044 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-60916aa6-d920-4595-b4f0-0f7642563b6c admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:03:45.312146 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-60916aa6-d920-4595-b4f0-0f7642563b6c admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.435539 Aug 07 01:03:45.312836 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 29/56] 10.222.0.26 () {64 vars in 1235 bytes} [Sun Aug 7 01:03:44 2022] POST /compute/v2.1/flavors => generated 405 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:03:46.320986 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-4b5af8bf-8fbc-48a5-b55d-bd82519b0e9d admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:46.322319 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-4b5af8bf-8fbc-48a5-b55d-bd82519b0e9d admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002439 Aug 07 01:03:46.323079 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 28/57] 10.222.0.26 () {56 vars in 1074 bytes} [Sun Aug 7 01:03:46 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:03:46.913545 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-78241470-3ac3-47a2-8866-bb6b363cf1f9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:03:46.937409 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-78241470-3ac3-47a2-8866-bb6b363cf1f9 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.474289 Aug 07 01:03:46.937965 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 30/58] 10.222.0.26 () {64 vars in 1235 bytes} [Sun Aug 7 01:03:46 2022] POST /compute/v2.1/flavors => generated 407 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:03:47.938358 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-4b5af8bf-8fbc-48a5-b55d-bd82519b0e9d admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:47.939386 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-4b5af8bf-8fbc-48a5-b55d-bd82519b0e9d admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001444 Aug 07 01:03:47.939770 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 29/59] 10.222.0.26 () {56 vars in 1074 bytes} [Sun Aug 7 01:03:47 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:03:48.535035 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-6c5bce9d-c1f1-4757-a2b8-add98bb2f6db admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:03:48.553877 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-6c5bce9d-c1f1-4757-a2b8-add98bb2f6db admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.465840 Aug 07 01:03:48.554436 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 31/60] 10.222.0.26 () {64 vars in 1235 bytes} [Sun Aug 7 01:03:48 2022] POST /compute/v2.1/flavors => generated 409 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:03:49.644102 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-4b5af8bf-8fbc-48a5-b55d-bd82519b0e9d admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:49.645614 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-4b5af8bf-8fbc-48a5-b55d-bd82519b0e9d admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001904 Aug 07 01:03:49.646290 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 30/61] 10.222.0.26 () {56 vars in 1074 bytes} [Sun Aug 7 01:03:49 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:03:50.315789 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-054d1af7-9e91-4fb4-990e-5155c486c9e9 admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:03:50.337383 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-054d1af7-9e91-4fb4-990e-5155c486c9e9 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.549284 Aug 07 01:03:50.337886 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 32/62] 10.222.0.26 () {60 vars in 1202 bytes} [Sun Aug 7 01:03:49 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:04:44.226384 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-4b5af8bf-8fbc-48a5-b55d-bd82519b0e9d admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:04:44.227396 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-4b5af8bf-8fbc-48a5-b55d-bd82519b0e9d admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001450 Aug 07 01:04:44.227795 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 31/63] 10.222.0.26 () {54 vars in 931 bytes} [Sun Aug 7 01:04:44 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:04:44.239184 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:04:44.637128 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-8b551fa1-0eb7-47fd-9624-6c6b71666ec8 admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:04:44.657474 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-8b551fa1-0eb7-47fd-9624-6c6b71666ec8 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.422727 Aug 07 01:04:44.658013 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 33/64] 10.222.0.26 () {60 vars in 1292 bytes} [Sun Aug 7 01:04:44 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:04:44.679090 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG dbcounter [-] [103441] Writing DB stats nova_api:SELECT=14 {{(pid=103441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:04:44.682773 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-8fb1f495-150b-4609-8f3a-46f24e67c57a admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:04:44.691290 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-8fb1f495-150b-4609-8f3a-46f24e67c57a admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013776 Aug 07 01:04:44.691835 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 32/65] 10.222.0.26 () {60 vars in 1283 bytes} [Sun Aug 7 01:04:44 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:04:44.704215 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-ffb8a06f-7b55-49eb-b0d8-f62d00a031eb admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:04:44.712151 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-ffb8a06f-7b55-49eb-b0d8-f62d00a031eb admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011535 Aug 07 01:04:44.712725 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 34/66] 10.222.0.26 () {60 vars in 1286 bytes} [Sun Aug 7 01:04:44 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:04:44.724760 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-f966676d-2175-4ae3-8ff4-0f9c045ab444 admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:04:44.733153 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-f966676d-2175-4ae3-8ff4-0f9c045ab444 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011935 Aug 07 01:04:44.733702 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 33/67] 10.222.0.26 () {60 vars in 1286 bytes} [Sun Aug 7 01:04:44 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:04:44.746227 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-34c6baf9-a8c6-4ca1-8e05-be0d2a2719fe admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:04:44.754025 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-34c6baf9-a8c6-4ca1-8e05-be0d2a2719fe admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012033 Aug 07 01:04:44.754544 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 35/68] 10.222.0.26 () {60 vars in 1283 bytes} [Sun Aug 7 01:04:44 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:07:46.571962 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-f966676d-2175-4ae3-8ff4-0f9c045ab444 admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:07:46.573030 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-f966676d-2175-4ae3-8ff4-0f9c045ab444 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001531 Aug 07 01:07:46.573436 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 34/69] 10.222.0.26 () {56 vars in 1074 bytes} [Sun Aug 7 01:07:46 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:07:46.721677 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writing DB stats nova_api:SELECT=6 {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:07:46.919700 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-1fb6a24a-d4df-41e8-8760-1d9043edcc9d admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:07:46.923609 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 01:07:46.923609 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: warnings.warn(msg) Aug 07 01:07:46.925666 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-1fb6a24a-d4df-41e8-8760-1d9043edcc9d admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:07:46.926487 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-1fb6a24a-d4df-41e8-8760-1d9043edcc9d admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:07:46.927242 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-1fb6a24a-d4df-41e8-8760-1d9043edcc9d admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:07:46.928520 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-1fb6a24a-d4df-41e8-8760-1d9043edcc9d admin admin] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:07:46.929292 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-1fb6a24a-d4df-41e8-8760-1d9043edcc9d admin admin] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:07:46.929836 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-1fb6a24a-d4df-41e8-8760-1d9043edcc9d admin admin] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:07:46.968887 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-1fb6a24a-d4df-41e8-8760-1d9043edcc9d admin admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.248673 Aug 07 01:07:46.969484 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 36/70] 10.222.0.26 () {60 vars in 1193 bytes} [Sun Aug 7 01:07:46 2022] GET /compute/v2.1/os-services => generated 791 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 07 01:07:47.856069 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-f966676d-2175-4ae3-8ff4-0f9c045ab444 admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:07:47.857109 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-f966676d-2175-4ae3-8ff4-0f9c045ab444 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001552 Aug 07 01:07:47.857921 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 35/71] 10.222.0.26 () {56 vars in 1074 bytes} [Sun Aug 7 01:07:47 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:07:48.082061 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-7961631d-9912-4b9a-bfd6-1042a0117ab6 admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:07:48.084794 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 01:07:48.084794 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: warnings.warn(msg) Aug 07 01:07:48.086053 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-7961631d-9912-4b9a-bfd6-1042a0117ab6 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:07:48.087032 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-7961631d-9912-4b9a-bfd6-1042a0117ab6 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:07:48.087032 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-7961631d-9912-4b9a-bfd6-1042a0117ab6 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:07:48.087854 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-7961631d-9912-4b9a-bfd6-1042a0117ab6 admin admin] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:07:48.088869 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-7961631d-9912-4b9a-bfd6-1042a0117ab6 admin admin] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:07:48.088869 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-7961631d-9912-4b9a-bfd6-1042a0117ab6 admin admin] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:07:48.111303 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-7961631d-9912-4b9a-bfd6-1042a0117ab6 admin admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.100188 Aug 07 01:07:48.111903 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 37/72] 10.222.0.26 () {60 vars in 1193 bytes} [Sun Aug 7 01:07:48 2022] GET /compute/v2.1/os-services => generated 791 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:07:59.480556 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-f966676d-2175-4ae3-8ff4-0f9c045ab444 admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:07:59.481555 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-f966676d-2175-4ae3-8ff4-0f9c045ab444 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001492 Aug 07 01:07:59.481953 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 36/73] 10.222.0.26 () {56 vars in 1074 bytes} [Sun Aug 7 01:07:59 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:07:59.625824 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writing DB stats nova_cell0:SELECT=4 {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:07:59.626617 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writing DB stats nova_cell1:SELECT=4 {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:07:59.627042 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG dbcounter [-] [103442] Writing DB stats nova_api:SELECT=8 {{(pid=103442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:07:59.685414 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-b0a8c175-12b4-4768-a907-610ac93419da admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:07:59.689897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 01:07:59.689897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: warnings.warn(msg) Aug 07 01:07:59.692051 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-b0a8c175-12b4-4768-a907-610ac93419da admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:07:59.692930 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-b0a8c175-12b4-4768-a907-610ac93419da admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:07:59.694312 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-b0a8c175-12b4-4768-a907-610ac93419da admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:07:59.695580 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-b0a8c175-12b4-4768-a907-610ac93419da admin admin] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:07:59.696421 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-b0a8c175-12b4-4768-a907-610ac93419da admin admin] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:07:59.697176 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-b0a8c175-12b4-4768-a907-610ac93419da admin admin] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:07:59.743575 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-b0a8c175-12b4-4768-a907-610ac93419da admin admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.90 time: 0.121994 Aug 07 01:07:59.744555 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 38/74] 10.222.0.26 () {60 vars in 1193 bytes} [Sun Aug 7 01:07:59 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:08:00.640120 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: DEBUG nova.api.openstack.wsgi [None req-f966676d-2175-4ae3-8ff4-0f9c045ab444 admin admin] Calling method '>' {{(pid=103441) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:08:00.641355 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: INFO nova.api.openstack.requestlog [None req-f966676d-2175-4ae3-8ff4-0f9c045ab444 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001712 Aug 07 01:08:00.641908 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: [pid: 103441|app: 0|req: 37/75] 10.222.0.26 () {56 vars in 1074 bytes} [Sun Aug 7 01:08:00 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:08:00.849425 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG nova.api.openstack.wsgi [None req-ce79abc3-8222-45d4-892e-faad59ad5072 admin admin] Calling method '>' {{(pid=103442) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:08:00.852575 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 01:08:00.852575 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: warnings.warn(msg) Aug 07 01:08:00.853727 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-ce79abc3-8222-45d4-892e-faad59ad5072 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:08:00.854200 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-ce79abc3-8222-45d4-892e-faad59ad5072 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:08:00.854623 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-ce79abc3-8222-45d4-892e-faad59ad5072 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:08:00.855348 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-ce79abc3-8222-45d4-892e-faad59ad5072 admin admin] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:08:00.855792 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-ce79abc3-8222-45d4-892e-faad59ad5072 admin admin] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:08:00.856205 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: DEBUG oslo_concurrency.lockutils [None req-ce79abc3-8222-45d4-892e-faad59ad5072 admin admin] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103442) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:08:00.879232 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: INFO nova.api.openstack.requestlog [None req-ce79abc3-8222-45d4-892e-faad59ad5072 admin admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.90 time: 0.087178 Aug 07 01:08:00.879897 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: [pid: 103442|app: 0|req: 39/76] 10.222.0.26 () {60 vars in 1193 bytes} [Sun Aug 7 01:08:00 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:09:00.330196 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: Sun Aug 7 01:09:00 2022 - graceful shutdown triggered... Aug 07 01:09:00.330929 dv-cinder-master-3-iscsi devstack@n-api.service[103442]: Gracefully killing worker 2 (pid: 103442)... Aug 07 01:09:00.331496 dv-cinder-master-3-iscsi devstack@n-api.service[103441]: Gracefully killing worker 1 (pid: 103441)... Aug 07 01:09:00.333063 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Aug 07 01:09:01.330412 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: worker 1 buried after 1 seconds Aug 07 01:09:01.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: worker 2 buried after 1 seconds Aug 07 01:09:01.330995 dv-cinder-master-3-iscsi devstack@n-api.service[103440]: goodbye to uWSGI. Aug 07 01:09:01.332120 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Aug 07 01:09:01.332698 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Aug 07 01:09:01.336336 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Aug 07 01:09:01.347476 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Aug 07 01:09:01.349768 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Aug 7 01:09:01 2022] *** Aug 07 01:09:01.349768 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 07 01:09:01.349768 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 07 01:09:01.349768 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: nodename: dv-cinder-master-3-iscsi Aug 07 01:09:01.349768 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: machine: x86_64 Aug 07 01:09:01.349768 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: clock source: unix Aug 07 01:09:01.349768 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: pcre jit disabled Aug 07 01:09:01.349768 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: detected number of CPU cores: 4 Aug 07 01:09:01.349768 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: current working directory: / Aug 07 01:09:01.349768 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: detected binary path: /usr/bin/uwsgi-core Aug 07 01:09:01.349768 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: your processes number limit is 47680 Aug 07 01:09:01.349768 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: your memory page size is 4096 bytes Aug 07 01:09:01.350560 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: detected max file descriptor number: 2048 Aug 07 01:09:01.350560 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: lock engine: pthread robust mutexes Aug 07 01:09:01.350560 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: thunder lock: enabled Aug 07 01:09:01.350560 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Aug 07 01:09:01.350560 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 07 01:09:01.391941 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: Python main interpreter initialized at 0x5572a5b219c0 Aug 07 01:09:01.391941 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: python threads support enabled Aug 07 01:09:01.391941 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: your server socket listen backlog is limited to 100 connections Aug 07 01:09:01.391941 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: your mercy for graceful operations on workers is 90 seconds Aug 07 01:09:01.392447 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: mapped 403077 bytes (393 KB) for 2 cores Aug 07 01:09:01.392447 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: *** Operational MODE: preforking *** Aug 07 01:09:01.392743 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: *** uWSGI is running in multiple interpreter mode *** Aug 07 01:09:01.392894 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: spawned uWSGI master process (pid: 117966) Aug 07 01:09:01.393218 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Aug 07 01:09:01.393964 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: spawned uWSGI worker 1 (pid: 117983, cores: 1) Aug 07 01:09:01.394447 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: spawned uWSGI worker 2 (pid: 117984, cores: 1) Aug 07 01:09:01.394738 dv-cinder-master-3-iscsi devstack@n-api.service[117966]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 07 01:09:02.117845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 07 01:09:02.159107 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 07 01:09:03.318848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 01:09:03.318848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: from cryptography.utils import int_from_bytes Aug 07 01:09:03.319546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 01:09:03.319546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: from cryptography.utils import int_from_bytes Aug 07 01:09:03.406653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 01:09:03.406653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: from cryptography.utils import int_from_bytes Aug 07 01:09:03.406653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 01:09:03.406653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: from cryptography.utils import int_from_bytes Aug 07 01:09:05.708873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 07 01:09:05.709443 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 07 01:09:05.709672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 07 01:09:05.710044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 07 01:09:05.710436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 07 01:09:05.710850 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.711176 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.711498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.711822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.712159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.712498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.712887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.713147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.713481 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.713746 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.714059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.714380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.714705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.715014 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.715324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.715646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.715952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.716308 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.716605 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.717109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.717556 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.717865 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.718178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.718520 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.718999 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.719183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.719456 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.719769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.720097 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.720439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.720848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.721169 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.721484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.721799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.722223 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.722542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.722853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.723181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.723497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.723875 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.724211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.724573 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.724891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.725216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.725517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.725824 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.726133 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.726455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.726757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.727067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.727324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.727632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.727955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.728372 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.728729 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.729120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.729275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.735776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.735776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.735776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.735776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.735776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.735776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.735776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.735776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.737081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.737081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.737081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.737081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.737081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.737081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.737081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.737081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.738272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.738272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.738272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.738272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.738272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.738272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.738272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.738272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.739440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.739440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.739440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.739440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.739440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.739440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.739440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.739440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.740281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.740281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.740281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.740522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.740835 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.741145 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.741451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.741897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.742209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.742565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.742896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.743207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.743517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.743827 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.744152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.744502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.744815 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.745900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.746058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.746343 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.747062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.747579 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.747895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.748321 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.748571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.748901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.749194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.749960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.749960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.750884 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.751338 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.752060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.752434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.752877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.753298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.753532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.753855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.754193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.754575 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.755408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.755736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.756047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.756403 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.756749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.757049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.757370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.757684 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.758008 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.758358 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.758692 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.759040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.759359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.759698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.760032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.760382 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.760698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.761204 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.761336 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.761657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.761973 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.762288 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.762608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.762930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.763263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.763587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.763907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.764225 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.764569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.764891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.765225 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.765553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.765867 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.766210 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.766559 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.766875 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.767229 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.767532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.767850 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.768165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.768504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.768833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.769173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.769489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.769813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.770128 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.770441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.770814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.771136 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.771455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.771804 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.772126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.772473 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.772787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.773184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.773596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.773779 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.774047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.774358 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.774667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.774976 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.775289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.775596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.775920 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.776278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.776583 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.776924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.777241 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.777556 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.777869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.778183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.778492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.778811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.779367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.779725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.780062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.780413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.780740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.781071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.781462 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.781675 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.781997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.782315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.782630 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.782946 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.783264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.783760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.783953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.785819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.785819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.785819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.785819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.785819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.785819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.786939 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.787024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.787290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.787604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.787914 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.788224 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.788612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.788892 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.789226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.789840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.789840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.790087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.790441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.790823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.791134 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.791445 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.791755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.792108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.792447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.792799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.793199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.793490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.793906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.794226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.794440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.794754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.795071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.795733 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.795733 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.796089 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.796375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.796661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.796983 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.797317 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.797635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.797956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.798265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.798587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.798934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.799283 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.799599 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.799921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.800261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.800591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.800904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.801228 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.801537 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.802214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.802337 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.802641 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.802958 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.803271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.803656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.804052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.804318 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.804768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.805091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.805509 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.805848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.806163 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.806704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.807033 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.807425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.807745 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.808055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.808430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.808890 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.809244 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.809602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.809962 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.810278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.810618 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.810911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.811493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.811736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.811947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.812296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.812733 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.813170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.813498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.813895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.814616 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.814772 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.815083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.815396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.815901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.816475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.816975 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.817334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.818203 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.818516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.818843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.819176 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.819495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.819825 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.820139 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.820482 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.820813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.821134 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.821652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.822166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.822489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.822806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.823126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.823460 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.823781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.824098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.824465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.824776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.825091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.825421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.826010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.826350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.826674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.827000 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.827475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.827654 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.828011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.828444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.828840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.829159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.829550 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.829924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.830277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.830596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.830912 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.831222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.831526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.832001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.832278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.832636 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.832811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.833250 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.833454 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.833763 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.834087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.834390 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.834726 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.835041 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.835349 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.835662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.835960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.836333 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.836643 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.836992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.837327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.837671 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.837981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.838426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.838620 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.838929 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.839729 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.839729 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.840199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.840199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.840821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.840821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.841210 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.841483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.841646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.841915 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.842230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.842530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.842908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.843143 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.843469 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.843747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.844084 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.844467 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.844687 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.845006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.845304 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.845623 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.845977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.846249 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.846621 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.846852 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.847166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.847483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.847799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.848120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.848523 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.849591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.849591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.849899 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.850179 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.850503 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.850805 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.851335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.851538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.851754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.852001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.853142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.853142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.853142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.854464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.854464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.854464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.854464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.854464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.855928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.855928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.855928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.855928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.855928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.857188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.857188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.857188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.857679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.857830 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 07 01:09:05.858619 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.859041 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 07 01:09:05.859313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.859313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.859890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 07 01:09:05.860054 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.860283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 07 01:09:05.860491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.860648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 07 01:09:05.860815 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.860975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.861127 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.861278 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.861427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.861583 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.861752 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.861905 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.862063 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.862063 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.862380 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.862580 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.862792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.862968 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.865345 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.865543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.865700 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.865868 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.866031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.866243 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.866402 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.866564 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.866716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.866890 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.866890 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.867196 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.867357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.867507 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.867656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.867820 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.867970 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.868175 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.868446 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.868446 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.868864 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.869175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.869335 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.869495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.869495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.869915 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.870226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.870430 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.870679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.870860 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.871031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.871031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.871339 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.871491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.871643 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.871792 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.871940 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.872088 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.872088 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.872493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.872694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.872694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.873152 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.873552 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.873552 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.873991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.874156 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.874865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.875086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.875648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.875883 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.876068 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.876230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.876504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.876668 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.876668 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.877003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.877156 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.877321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.877472 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.877630 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.877854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.877854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.878168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.878327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.878327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.878631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.878782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.878782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.879082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.879375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.879375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.879972 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.880146 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.880146 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.880649 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.880887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.880887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.881339 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.881505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.881665 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.881818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.881990 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.882170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.882170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.882531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.882712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.882712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.883138 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.883301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.883473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.883738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.883738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.884039 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.884309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.884309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.885125 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.885376 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.885376 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.886138 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.886440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.886440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.886872 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.887101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.887317 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.887533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.887533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.887964 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.888175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.888618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.888812 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.888981 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.889141 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.889294 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.889585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.889744 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.889925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.889925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.890230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.890230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.890671 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.890852 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.891082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.891293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.891506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.891714 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.891937 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.892094 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.892352 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.892514 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.892674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.892825 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.893007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.893180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.893344 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.893502 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.893662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.893955 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.894143 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.894374 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.894582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.894800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.895272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.895272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.895637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.895952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.895952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.896614 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.896797 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.897091 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.897288 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.897528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.897780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.898000 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.898212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.898433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.898655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.898881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.899071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.899952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.900219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.900426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.900666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.901034 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.901261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.901422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.901573 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.901573 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.901953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.902108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.902348 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.902704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.902878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.903049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.903277 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.903446 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.903599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.903750 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.903923 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.904194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.904395 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.904550 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.904708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.904708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.905022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.905207 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.905655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.905814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.906086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.906255 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.906409 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.906971 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.907180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.907373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.907373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.907780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.908061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.908061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.908570 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.908801 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.909040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.909260 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.909484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.909708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.909708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.910267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.910483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.910874 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.913227 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.913512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.913683 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.913840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.913840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.914168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.914453 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.914630 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.914854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.915050 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.915232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.915407 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.915841 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.916009 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.916363 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.916536 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.916770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.916943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.917261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.917426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.917587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.917750 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.917901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.917901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.918186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.918338 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.918504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.918668 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.918829 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.918988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.919146 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.919297 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.919459 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.919611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.919784 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.919939 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.920089 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.920272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.920546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.920744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.920910 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.921063 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.921261 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.921419 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.921579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.921762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.921981 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.922145 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.922432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.922592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.922760 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.922913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.923100 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.923432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.923669 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.923932 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.924122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.924122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.924611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.924802 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.924957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.925120 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.925273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.925431 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.925582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.925958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.926127 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.926299 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.926603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.927486 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.927767 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.927924 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.928076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.928307 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 01:09:05.928477 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.928629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.928801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.928801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.929116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.929278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.929436 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.929436 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.930308 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.930475 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.930766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.930926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.931153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.931385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.931574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.931745 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.931926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.932079 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.932230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.932439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.932735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.932913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.933066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.933221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.933371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.933523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.933698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.933932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.934088 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.934240 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.934390 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.934541 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.934840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.935001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.935152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.935311 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.935463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.935614 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.935773 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.936161 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.936369 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.936551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.936821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.937038 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.937285 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.937441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.937593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.937763 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.938061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.938228 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.938388 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.938549 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.938709 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.938881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.939033 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.939454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.939615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.939768 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.939918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.940069 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.940218 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.940468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.940468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.940799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.941057 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.941236 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.941388 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.941682 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.941857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.942018 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.942170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.942320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.942582 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.942875 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.943035 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.943186 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.943337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.943503 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.943681 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.943843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.944004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.944179 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.944354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.944516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.944676 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.944855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.944855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.945257 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.945409 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.945558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.945799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.945972 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.946359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.946528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.946680 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.946863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.947169 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.947337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.947337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.947642 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.947803 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.947968 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.948378 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.948632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.948887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.949110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.949343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.949560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.949717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.949868 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.950035 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.950265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.950506 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.950763 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.951009 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.951596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.951864 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.952113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.952403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.952646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.952921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.953302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.953563 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.953811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.954075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.954323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.954560 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.954817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.955052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.955454 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.955712 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.955967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.956216 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.957949 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.957949 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.958529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.958529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.958859 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.958859 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.959158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.959309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.959471 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.959653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.959815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.959967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.960118 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.960290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.960467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.960627 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.960822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.960977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.961136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.961294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.961577 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.961739 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.961903 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.962062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.962214 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.962363 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.962527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.962679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.962837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.962988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.963137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.963426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.963652 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.963906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.964135 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.964353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.964523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.964693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.964894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.965052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.965203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.965354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.965739 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.965925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.966078 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.966252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.966416 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.966721 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.966989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.966989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.967364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.967573 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.967810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.968079 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.968351 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.968547 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.968911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.969075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.969075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.969075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.969541 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.969702 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.969861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.970019 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.970019 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.970307 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.970467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.970619 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.970772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.970960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.971113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.971275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.971449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.971711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.971869 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.972028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.972190 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.972399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.972554 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.972722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.972880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.973029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.973189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.973351 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.974225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.974404 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.974557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.974716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.974868 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.975031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.975192 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.975359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.975594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.975912 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.976073 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.976254 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.976254 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.976838 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.977114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.977283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.977446 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.977606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.977815 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.977974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.978241 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.978437 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.978652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.978808 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.979105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.979414 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.979597 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.979749 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.979909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.980070 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.980222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.980425 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.980585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.980763 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.980936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.981097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.981256 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.981415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.981574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.981869 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.982035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.982188 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.982338 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.982489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.982639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.982802 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.982955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.983105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.983256 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.983406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.983406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.983834 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.984013 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.984182 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.984362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.984517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.984761 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.984925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.985078 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.985238 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.985393 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.985552 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.985714 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.985887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.986118 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.986273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.986273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.986579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.986739 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.987039 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.987335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.987494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.987494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.987805 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.988017 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.988182 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.988489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.988662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.988834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.988834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.989142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.990190 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.990359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.990511 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.990678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.990830 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.990999 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.991152 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.991303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.991476 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.991639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.991791 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.992080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.992321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.992483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.992653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.992806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.992957 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.993182 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.993349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.993510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.993676 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.993988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.994157 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.994310 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.994469 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.994620 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.994784 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.994936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.995087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.995246 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.995246 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.995567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.995719 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.995878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.996051 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.996202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.996392 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.996556 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.996718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.996870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.997040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.997344 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.997624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.997818 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.997976 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.998195 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.998347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.998642 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.998808 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.998970 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.999130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.999282 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.999432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.999584 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.999749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:05.999910 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.000089 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.000089 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.000720 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.000882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.001065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.001226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.001404 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.001562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.001713 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.001864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.002024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.002309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.002467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.002630 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.002783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.002947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.003129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.003289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.003449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.003742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.003902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.004201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.004386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.004538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.004538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.005450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.005624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.005787 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.005956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.006134 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.006134 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.006431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.006431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.007540 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.007852 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.008064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.008320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.008494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.008664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.008951 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.009113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.009113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.009463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.009615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.009776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.009930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.010092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.010245 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.010443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.010736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.011054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.011212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.011371 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.011530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.011696 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.011854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.012024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.012178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.012481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.012649 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.012802 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.012954 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.013114 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.013360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.013517 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.013684 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.013863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.013863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.014180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.014453 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.014618 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.014780 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.014940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.015091 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.015324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.015487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.015648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.015800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.015964 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.016129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.016363 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.016523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.016685 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.016685 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.016974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.017125 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.017284 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.017436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.017749 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.018055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117983) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 07 01:09:06.018296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.018296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.018296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.018296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.018296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.018296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.018296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.018296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.019600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.019600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.019600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.019600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.019600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.019600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.019600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.019600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.022001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.022001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.022001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.022001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.022001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.022001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.022001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.022001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.023947 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.023947 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.023947 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.023947 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.023947 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.023947 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.023947 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.023947 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.026136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.026136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.026136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.026136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.026136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.026136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.026136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.026136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.028599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.028599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.028599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.028599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.028599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.028599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.028599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.028599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.030362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.030362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.030362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.030362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.030362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.030362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.030362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.030362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.032645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.032645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.032645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.032645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.032645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.032645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.032645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.032645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.034843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.034843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.034843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.034843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.034843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.034843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.034843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.034843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.036880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.036880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.036880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.036880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.036880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.036880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.036880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.036880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.038158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.038158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.038158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.038158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.038158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.038158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.038158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.038158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.040418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.040418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.040418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.040418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.040418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.040418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.040418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.040418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.041902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.041902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.041902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.041902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.041902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.041902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.041902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.041902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.043488 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.043488 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.043488 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.043488 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.043488 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.043488 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.043488 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.043488 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.044658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.044658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.044658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.044658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.044658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.044658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.044658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.044658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.045966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.045966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.045966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.045966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.045966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.045966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.045966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.045966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.047238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.047238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.047238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.047238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.047238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.047238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.047238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.047238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.048522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.048522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.048522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.048522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.048522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.048522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.048522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.048522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.050285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.050285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.050285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.050285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.050285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.050285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.050285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.050285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.051623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.051623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.051623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.051623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.051623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.051623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.051623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.051623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.053200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.053200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.053200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.053200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.053200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.053200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.053200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.053200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.054473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.054473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.054473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.054473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.054473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.054473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.054473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.054473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.055954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.055954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.055954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.055954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.055954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.055954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.055954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.055954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.057607 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.057607 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.057607 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.057607 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.057607 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.057607 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.057607 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.057607 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.058854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.058854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.058854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.058854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.058854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.058854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.058854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.060392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.060392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.060392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.060392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.060392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.061156 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.061156 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.061156 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.062525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.062525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.062525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.062525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.063294 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.063294 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.063717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.063875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.064496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.064842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.065169 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.065498 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.065877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.066217 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.066585 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.066913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.067239 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.068075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.068440 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.068776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.069105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.069742 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.070024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.070405 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.070667 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.070960 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.071300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.071645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.071978 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.072481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.072734 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.073196 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.073468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.073748 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.074140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.074398 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.075079 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.075079 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.076665 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.076665 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.076665 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.077668 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.077668 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.078162 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.078162 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.078610 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.078860 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.079112 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.079456 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.080049 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.080218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.080685 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.081495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.081495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.081809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.082098 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.082437 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.082861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.084847 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.085591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.085836 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.086565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.086565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.087857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.087857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.087857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.087857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.088953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.088953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.088953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.088953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.090253 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.090253 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.090253 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.091105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.091105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.091105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.091890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.092044 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.092425 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.092998 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.093213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.093506 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.093875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.094419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.094581 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.094885 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.095246 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.095826 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.095998 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.096308 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.096848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.097044 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.097380 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.098483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.098483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.098483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.099309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.099309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.099783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.100025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.100609 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.100609 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.101111 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.101354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.101659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.102355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.102646 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.102881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.103159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.103431 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.103790 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.104151 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.104509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.104852 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.105172 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.105545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.105879 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.106204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.106533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.106855 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.107176 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.107542 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.107871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.108518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.108954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.109273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.109653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.109998 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.110327 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.110646 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.110964 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.111300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.111621 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.111944 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.112349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.112598 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.113009 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.113231 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.113552 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.113903 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.114771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.115158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.115513 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.115878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.116295 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.116625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.116993 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.117349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.117683 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.118013 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.118342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.118670 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.118993 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.119313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.119664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.120009 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.120380 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.120757 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.121082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.121403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.121845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.122171 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.122502 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.122835 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.123152 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.123485 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.123816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.124140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.124492 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.124804 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.125247 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.125493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.125811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.126240 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.126532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.126763 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.127064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.127391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.127717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.128037 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.128374 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.128691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.129010 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.129324 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.129670 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.130054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.131056 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.131253 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.131590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.131927 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.132441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.132785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.133100 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.133416 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.133738 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.134052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.134367 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.134686 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.134997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.135337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.135719 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.136111 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.137236 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.137236 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.137519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.137671 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.137942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.138267 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.138598 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.138928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.139249 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.139631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.139993 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.140360 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.140691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.141015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.141337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.141657 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.141995 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.142332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.142647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.142982 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.143309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.143640 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.143968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.144314 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.144652 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.144979 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.145294 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.145648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.145968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.146288 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.146620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.146973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.147242 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.147567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.147894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.148230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.148582 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.148903 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.149239 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.149574 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.149931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.150254 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.150619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.150951 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.151281 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.151593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.151895 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 01:09:06.152384 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117984) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 07 01:09:06.156549 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO dbcounter [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] Registered counter for database nova_api Aug 07 01:09:06.192271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_db.sqlalchemy.engines [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117983) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:09:06.201122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writer thread running {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:09:06.263886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.context [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dfea601e-2ab8-4cc9-8a98-b642673a3d20(cell1) {{(pid=117983) load_cells /opt/stack/nova/nova/context.py:464}} Aug 07 01:09:06.268269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:09:06.268762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:09:06.270384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:09:06.271943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:09:06.272352 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:09:06.274493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:09:06.276791 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO dbcounter [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] Registered counter for database nova_cell0 Aug 07 01:09:06.281533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO dbcounter [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] Registered counter for database nova_cell1 Aug 07 01:09:06.290260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_db.sqlalchemy.engines [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117983) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:09:06.291840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_db.sqlalchemy.engines [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117983) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:09:06.299048 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writer thread running {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:09:06.300414 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writer thread running {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:09:06.313131 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO dbcounter [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] Registered counter for database nova_api Aug 07 01:09:06.335231 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_db.sqlalchemy.engines [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117984) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:09:06.343070 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writer thread running {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:09:06.401026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.context [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),dfea601e-2ab8-4cc9-8a98-b642673a3d20(cell1) {{(pid=117984) load_cells /opt/stack/nova/nova/context.py:464}} Aug 07 01:09:06.405253 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:09:06.405711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:09:06.407368 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:09:06.408305 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:09:06.408698 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:09:06.410785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:09:06.413506 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO dbcounter [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] Registered counter for database nova_cell0 Aug 07 01:09:06.417890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO dbcounter [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] Registered counter for database nova_cell1 Aug 07 01:09:06.427144 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_db.sqlalchemy.engines [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117984) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:09:06.427732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_db.sqlalchemy.engines [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117984) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:09:06.434737 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writer thread running {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:09:06.436190 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writer thread running {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:09:06.449904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO dbcounter [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] Registered counter for database nova_cell0 Aug 07 01:09:06.461445 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_db.sqlalchemy.engines [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117983) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:09:06.468336 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writer thread running {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:09:06.588280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO dbcounter [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] Registered counter for database nova_cell0 Aug 07 01:09:06.596293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_db.sqlalchemy.engines [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117984) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 01:09:06.602337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writer thread running {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 01:09:06.799706 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: WARNING keystonemiddleware.auth_token [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 07 01:09:06.807882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: WARNING keystonemiddleware.auth_token [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 07 01:09:06.936523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: WARNING keystonemiddleware.auth_token [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 07 01:09:06.945444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: WARNING keystonemiddleware.auth_token [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 07 01:09:07.197285 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: WARNING keystonemiddleware.auth_token [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 07 01:09:07.205459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: WARNING keystonemiddleware.auth_token [None req-9eaba514-868c-4882-91b8-64b7096904bc None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 07 01:09:07.214100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5572a5b219c0 pid: 117983 (default app) Aug 07 01:09:07.311777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: WARNING keystonemiddleware.auth_token [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 07 01:09:07.319013 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: WARNING keystonemiddleware.auth_token [None req-8f713b26-111f-4faf-bdd9-df78cd8b2125 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 07 01:09:07.325475 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5572a5b219c0 pid: 117984 (default app) Aug 07 01:09:53.127038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=3 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:09:53.127038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=3 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:09:53.127038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=3 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:09:53.127038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=3 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:09:55.784713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-98ecad62-10e4-4ba0-b3ab-54faf83876b2 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Action: 'create', calling method: .version_select at 0x7f1faf310dc0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1323464361"}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:09:58.101029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-98ecad62-10e4-4ba0-b3ab-54faf83876b2 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 5.081859 Aug 07 01:09:58.102543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1/1] 10.222.0.26 () {60 vars in 1155 bytes} [Sun Aug 7 01:09:53 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 5087 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Aug 07 01:09:58.151789 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5d85161b-0d88-427e-b925-e4c88c6f43bf tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf18ac10>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:09:58.260845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5d85161b-0d88-427e-b925-e4c88c6f43bf tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc" status: 200 len: 345 microversion: 2.1 time: 0.131029 Aug 07 01:09:58.262844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 2/2] 10.222.0.26 () {58 vars in 1245 bytes} [Sun Aug 7 01:09:58 2022] GET /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc => generated 345 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:09:58.519544 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-48b727ba-d483-413c-ba78-527d80b5f165 tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] Calling method '.version_select at 0x7f1faf3e9940>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:09:58.565623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=3 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:09:58.567406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=3 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:09:58.569886 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=3 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:09:58.571375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=3 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:09:59.123403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-48b727ba-d483-413c-ba78-527d80b5f165 tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/fd40af11-e83e-4e75-b071-fae53261242e" status: 200 len: 351 microversion: 2.1 time: 1.389124 Aug 07 01:09:59.137415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 1/3] 10.222.0.26 () {58 vars in 1245 bytes} [Sun Aug 7 01:09:57 2022] GET /compute/v2.1/os-volumes/fd40af11-e83e-4e75-b071-fae53261242e => generated 351 bytes in 1408 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:09:59.294029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-800c583f-4998-474a-95e9-dcc6a66b4dd9 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf1a2310>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:00.104013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-800c583f-4998-474a-95e9-dcc6a66b4dd9 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc" status: 200 len: 346 microversion: 2.1 time: 0.826944 Aug 07 01:10:00.105794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 3/4] 10.222.0.26 () {58 vars in 1245 bytes} [Sun Aug 7 01:09:59 2022] GET /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc => generated 346 bytes in 830 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:00.134283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5b28dfd3-67ce-4be5-b855-1ac586035a2c tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Action: 'create', calling method: .version_select at 0x7f1faf238c10>, body: {"snapshot": {"volume_id": "679843ff-5657-404a-a0d0-aafd56316ffc", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1375149599"}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:10:00.758567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5b28dfd3-67ce-4be5-b855-1ac586035a2c tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.635878 Aug 07 01:10:00.759822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 2/5] 10.222.0.26 () {60 vars in 1162 bytes} [Sun Aug 7 01:10:00 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 640 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:10:00.781839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2c07df4f-495d-46eb-b457-544be490f28d tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf1a29d0>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:00.875322 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2c07df4f-495d-46eb-b457-544be490f28d tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca" status: 200 len: 287 microversion: 2.1 time: 0.103444 Aug 07 01:10:00.877192 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 4/6] 10.222.0.26 () {58 vars in 1251 bytes} [Sun Aug 7 01:10:00 2022] GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca => generated 287 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:00.893636 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4719f46c-06bd-475a-805a-0dffb2cf1019 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf1849d0>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:01.141588 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4719f46c-06bd-475a-805a-0dffb2cf1019 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.253185 Aug 07 01:10:01.157385 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-655d4cd8-e32d-4a18-970b-a564979959cc tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf18aca0>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:01.161725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 3/7] 10.222.0.26 () {58 vars in 1140 bytes} [Sun Aug 7 01:10:00 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:01.230969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-655d4cd8-e32d-4a18-970b-a564979959cc tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca" status: 200 len: 287 microversion: 2.1 time: 0.077788 Aug 07 01:10:01.242127 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 5/8] 10.222.0.26 () {58 vars in 1251 bytes} [Sun Aug 7 01:10:01 2022] GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca => generated 287 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:10:02.252023 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f3b8c418-8b87-43cb-946f-7ef663b1b36c tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf373d30>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:02.314171 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f3b8c418-8b87-43cb-946f-7ef663b1b36c tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca" status: 200 len: 287 microversion: 2.1 time: 0.067709 Aug 07 01:10:02.314959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 4/9] 10.222.0.26 () {58 vars in 1251 bytes} [Sun Aug 7 01:10:02 2022] GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca => generated 287 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:10:03.333752 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-570e21dd-2096-4bc3-971e-c39d712dcfcf tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf1a2940>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:03.513299 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-570e21dd-2096-4bc3-971e-c39d712dcfcf tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca" status: 200 len: 287 microversion: 2.1 time: 0.187086 Aug 07 01:10:03.515993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 6/10] 10.222.0.26 () {58 vars in 1251 bytes} [Sun Aug 7 01:10:03 2022] GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca => generated 287 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:04.532457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5f3bc5c9-9398-468a-a4f9-d104867b10fd tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf19e040>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:04.760080 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5f3bc5c9-9398-468a-a4f9-d104867b10fd tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca" status: 200 len: 287 microversion: 2.1 time: 0.227409 Aug 07 01:10:04.761028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 5/11] 10.222.0.26 () {58 vars in 1251 bytes} [Sun Aug 7 01:10:04 2022] GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca => generated 287 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:05.784315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-82702422-319f-4afe-ad20-a19c79a8ba30 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf1a2280>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:05.922226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-82702422-319f-4afe-ad20-a19c79a8ba30 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca" status: 200 len: 287 microversion: 2.1 time: 0.147642 Aug 07 01:10:05.923904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 7/12] 10.222.0.26 () {58 vars in 1251 bytes} [Sun Aug 7 01:10:05 2022] GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca => generated 287 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:06.944524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-76aa398a-30a1-48c6-9c66-ff3202409927 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf19e700>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:07.080105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-76aa398a-30a1-48c6-9c66-ff3202409927 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca" status: 200 len: 288 microversion: 2.1 time: 0.143126 Aug 07 01:10:07.092563 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 6/13] 10.222.0.26 () {58 vars in 1251 bytes} [Sun Aug 7 01:10:06 2022] GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca => generated 288 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:07.108584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-60f1fd27-75a9-44ca-8001-14e127a81e03 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf1ac1f0>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:07.298731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-60f1fd27-75a9-44ca-8001-14e127a81e03 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "DELETE /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca" status: 202 len: 0 microversion: 2.1 time: 0.205192 Aug 07 01:10:07.299921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 8/14] 10.222.0.26 () {58 vars in 1254 bytes} [Sun Aug 7 01:10:07 2022] DELETE /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca => generated 0 bytes in 207 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:10:07.312873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bee5504e-47a7-4c2f-9a8a-5665567f304c tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf1a2160>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:07.459413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bee5504e-47a7-4c2f-9a8a-5665567f304c tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca" status: 200 len: 287 microversion: 2.1 time: 0.150835 Aug 07 01:10:07.460383 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 9/15] 10.222.0.26 () {58 vars in 1251 bytes} [Sun Aug 7 01:10:07 2022] GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca => generated 287 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:10:07.606283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ddbcfb42-4865-40bd-b3b3-d5554fafe7aa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Action: 'create', calling method: .version_select at 0x7f1fafb13040>, body: {"keypair": {"name": "tempest-keypair-7392549", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEWFhRQgGTSgCmCFJJdq43/3WTDIP2UtGvcs2YN7PjxdpD/Duq5TWbUuZXqQRnuIxfdBjGjN9J3jV72XICWA/6j6t7DHW7+OF8DUVS8kSFr6UJxfrPsJ4PDIDw1k2YI8Zg=="}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:10:07.757109 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-ddbcfb42-4865-40bd-b3b3-d5554fafe7aa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Getting quotas for project 1b1d33b8c71e41e6bf23d4ad8590d2c2. Resources: {'key_pairs'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:10:07.776689 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-ddbcfb42-4865-40bd-b3b3-d5554fafe7aa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Getting quotas for user 5fafe42957914b929f9ffee53ab6e7ac and project 1b1d33b8c71e41e6bf23d4ad8590d2c2. Resources: {'key_pairs'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:10:08.014797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ddbcfb42-4865-40bd-b3b3-d5554fafe7aa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 398 microversion: 2.20 time: 0.882525 Aug 07 01:10:08.044625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 7/16] 10.222.0.26 () {62 vars in 1200 bytes} [Sun Aug 7 01:10:07 2022] POST /compute/v2.1/os-keypairs => generated 398 bytes in 890 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Aug 07 01:10:08.064592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f6b3403b-237c-479f-8627-980ce4405657 tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] Calling method '.version_select at 0x7f1faf1a2820>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:08.221473 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f6b3403b-237c-479f-8627-980ce4405657 tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/078d1b40-44cf-4cd5-88ac-5f1a75394c0f" status: 200 len: 350 microversion: 2.1 time: 0.163446 Aug 07 01:10:08.222488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 10/17] 10.222.0.26 () {58 vars in 1245 bytes} [Sun Aug 7 01:10:08 2022] GET /compute/v2.1/os-volumes/078d1b40-44cf-4cd5-88ac-5f1a75394c0f => generated 350 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:10:08.481001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e3732318-88b1-4cc7-be09-fe6323a1e7e6 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1fae919310>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:08.632388 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e3732318-88b1-4cc7-be09-fe6323a1e7e6 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca" status: 200 len: 287 microversion: 2.1 time: 0.159282 Aug 07 01:10:08.643413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 8/18] 10.222.0.26 () {58 vars in 1251 bytes} [Sun Aug 7 01:10:08 2022] GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca => generated 287 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:09.650171 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fe1a78e6-a1da-4330-90c6-1179947e6c6b tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf18aaf0>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:09.742177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fe1a78e6-a1da-4330-90c6-1179947e6c6b tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca" status: 200 len: 287 microversion: 2.1 time: 0.096418 Aug 07 01:10:09.743844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 11/19] 10.222.0.26 () {58 vars in 1251 bytes} [Sun Aug 7 01:10:09 2022] GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca => generated 287 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:10.761512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-11815694-7604-49eb-9fe9-cc1ae9c7a776 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf1688b0>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:10.824938 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-11815694-7604-49eb-9fe9-cc1ae9c7a776 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] HTTP exception thrown: Snapshot cc3cfb4d-8427-4280-8389-836d396b02ca could not be found. Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-11815694-7604-49eb-9fe9-cc1ae9c7a776 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Request method failure captured: Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca HTTP/1.1 Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xDgs7rHcX-oc9QApK-Hug_aEXBc7reGvuGY3JykitLnQYDia8SOvTO_avJvDbMpNt5wSK_F4H4yl2NbOR6sOTP8UbgR8CIC825kUlEi7ls0399yjBId4cqSwOitPvEITF2_SoJ-lBlrirMhSTXinE_dnhYQ8ZiUR1K8OHQW8S60RGU Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: fce127d62e6648d89d54ab21bd3b14e2 Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-125094030 Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:10:10.829546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:10:10.831871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:10:10.831871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-VolumesSnapshotsTestJSON-125094030 Aug 07 01:10:10.831871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: fce127d62e6648d89d54ab21bd3b14e2 Aug 07 01:10:10.831871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-125094030 Aug 07 01:10:10.831871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-VolumesSnapshotsTestJSON-125094030-project Aug 07 01:10:10.831871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:10:10.831871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:10:10.831871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: c59b392a2e1a4f32bba1433afc519d5e Aug 07 01:10:10.831871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-VolumesSnapshotsTestJSON-125094030-project Aug 07 01:10:10.831871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: Aug 07 01:10:10.831871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Snapshot cc3cfb4d-8427-4280-8389-836d396b02ca could not be found. Aug 07 01:10:10.831871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': 'cc3cfb4d-8427-4280-8389-836d396b02ca'} Aug 07 01:10:10.831871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot cc3cfb4d-8427-4280-8389-836d396b02ca could not be found. (HTTP 404) (Request-ID: req-764271c3-334e-4724-87d1-419ff55b1e0c) Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:10.833991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 07 01:10:10.836410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot cc3cfb4d-8427-4280-8389-836d396b02ca could not be found. Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Aug 07 01:10:10.837928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:10:10.839817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot cc3cfb4d-8427-4280-8389-836d396b02ca could not be found. Aug 07 01:10:10.839817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:10.839817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:10:10.839817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:10.839817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:10.839817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:10:10.839817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:10:10.839817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:10:10.839817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:10:10.839817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot cc3cfb4d-8427-4280-8389-836d396b02ca could not be found. Aug 07 01:10:10.839817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:10.839817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-11815694-7604-49eb-9fe9-cc1ae9c7a776 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Returning 404 to user: Snapshot cc3cfb4d-8427-4280-8389-836d396b02ca could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:10:10.843709 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-11815694-7604-49eb-9fe9-cc1ae9c7a776 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca" status: 404 len: 111 microversion: 2.1 time: 0.080160 Aug 07 01:10:10.844961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 9/20] 10.222.0.26 () {58 vars in 1251 bytes} [Sun Aug 7 01:10:10 2022] GET /compute/v2.1/os-snapshots/cc3cfb4d-8427-4280-8389-836d396b02ca => generated 111 bytes in 90 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 07 01:10:10.853950 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ee8de435-1526-4639-b192-e9a837097e49 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf1ac430>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:11.107659 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ee8de435-1526-4639-b192-e9a837097e49 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc" status: 202 len: 0 microversion: 2.1 time: 0.257438 Aug 07 01:10:11.108991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 12/21] 10.222.0.26 () {58 vars in 1248 bytes} [Sun Aug 7 01:10:10 2022] DELETE /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc => generated 0 bytes in 260 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 07 01:10:11.126112 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1dc65f75-7efe-4dc3-abc3-50c48befff7b tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf238ca0>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:11.450941 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1dc65f75-7efe-4dc3-abc3-50c48befff7b tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc" status: 200 len: 345 microversion: 2.1 time: 0.333929 Aug 07 01:10:11.452039 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 10/22] 10.222.0.26 () {58 vars in 1245 bytes} [Sun Aug 7 01:10:11 2022] GET /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc => generated 345 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:11.769801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6d7886ea-5d2c-428a-8f8f-973a812c6f45 tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] Calling method '.version_select at 0x7f1faf18adc0>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:11.934426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6d7886ea-5d2c-428a-8f8f-973a812c6f45 tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/d83fe0c6-0e85-4449-a1e0-58655b833c0f" status: 200 len: 351 microversion: 2.1 time: 0.169593 Aug 07 01:10:11.935897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 13/23] 10.222.0.26 () {58 vars in 1245 bytes} [Sun Aug 7 01:10:11 2022] GET /compute/v2.1/os-volumes/d83fe0c6-0e85-4449-a1e0-58655b833c0f => generated 351 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:11.962218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3b76a578-8c41-4746-b579-704e1b20ef4a tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] Calling method '.version_select at 0x7f1fae8ed700>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:12.476536 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3b76a578-8c41-4746-b579-704e1b20ef4a tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] 10.222.0.26 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.522690 Aug 07 01:10:12.502165 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 11/24] 10.222.0.26 () {58 vars in 1134 bytes} [Sun Aug 7 01:10:11 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:12.509689 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f982bd26-ea6c-4c56-833a-77cbe6ecde1a tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1faf1a2790>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:12.532077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ec6edb72-3918-44b3-8a69-82e1c089d2f9 tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] Calling method '.version_select at 0x7f1fae88b940>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:12.702718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-f982bd26-ea6c-4c56-833a-77cbe6ecde1a tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] HTTP exception thrown: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f982bd26-ea6c-4c56-833a-77cbe6ecde1a tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Request method failure captured: Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc HTTP/1.1 Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xDgs7rHcX-oc9QApK-Hug_aEXBc7reGvuGY3JykitLnQYDia8SOvTO_avJvDbMpNt5wSK_F4H4yl2NbOR6sOTP8UbgR8CIC825kUlEi7ls0399yjBId4cqSwOitPvEITF2_SoJ-lBlrirMhSTXinE_dnhYQ8ZiUR1K8OHQW8S60RGU Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: fce127d62e6648d89d54ab21bd3b14e2 Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-125094030 Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:10:12.710876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:10:12.712122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:10:12.712122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-VolumesSnapshotsTestJSON-125094030 Aug 07 01:10:12.712122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: fce127d62e6648d89d54ab21bd3b14e2 Aug 07 01:10:12.712122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-125094030 Aug 07 01:10:12.712122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-VolumesSnapshotsTestJSON-125094030-project Aug 07 01:10:12.712122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:10:12.712122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:10:12.712122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: c59b392a2e1a4f32bba1433afc519d5e Aug 07 01:10:12.712122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-VolumesSnapshotsTestJSON-125094030-project Aug 07 01:10:12.712122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: Aug 07 01:10:12.712122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:12.712122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '679843ff-5657-404a-a0d0-aafd56316ffc'} Aug 07 01:10:12.712122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. (HTTP 404) (Request-ID: req-f3c0de90-141a-4036-a71b-581d6a9bd606) Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:10:12.712921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 07 01:10:12.713893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 07 01:10:12.714814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:10:12.715762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:12.715762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:10:12.715762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:10:12.715762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:10:12.715762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:12.715762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:10:12.715762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:10:12.715762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:10:12.715762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:10:12.715762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:12.715762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:10:12.718795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f982bd26-ea6c-4c56-833a-77cbe6ecde1a tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Returning 404 to user: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:10:12.721642 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f982bd26-ea6c-4c56-833a-77cbe6ecde1a tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc" status: 404 len: 109 microversion: 2.1 time: 0.253552 Aug 07 01:10:12.736843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 14/25] 10.222.0.26 () {58 vars in 1245 bytes} [Sun Aug 7 01:10:12 2022] GET /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc => generated 109 bytes in 271 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 07 01:10:12.870563 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ec6edb72-3918-44b3-8a69-82e1c089d2f9 tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] 10.222.0.26 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.362678 Aug 07 01:10:12.882509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 12/26] 10.222.0.26 () {58 vars in 1149 bytes} [Sun Aug 7 01:10:12 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:12.888885 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-03732a50-3ec5-4d1e-a190-7076bccc0546 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1fae88b670>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:12.982711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-03732a50-3ec5-4d1e-a190-7076bccc0546 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] HTTP exception thrown: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-03732a50-3ec5-4d1e-a190-7076bccc0546 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Request method failure captured: Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: DELETE /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc HTTP/1.1 Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xDgs7rHcX-oc9QApK-Hug_aEXBc7reGvuGY3JykitLnQYDia8SOvTO_avJvDbMpNt5wSK_F4H4yl2NbOR6sOTP8UbgR8CIC825kUlEi7ls0399yjBId4cqSwOitPvEITF2_SoJ-lBlrirMhSTXinE_dnhYQ8ZiUR1K8OHQW8S60RGU Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: fce127d62e6648d89d54ab21bd3b14e2 Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-125094030 Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:10:12.986104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:10:12.987197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:10:12.987197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-VolumesSnapshotsTestJSON-125094030 Aug 07 01:10:12.987197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: fce127d62e6648d89d54ab21bd3b14e2 Aug 07 01:10:12.987197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-125094030 Aug 07 01:10:12.987197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-VolumesSnapshotsTestJSON-125094030-project Aug 07 01:10:12.987197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:10:12.987197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:10:12.987197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: c59b392a2e1a4f32bba1433afc519d5e Aug 07 01:10:12.987197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-VolumesSnapshotsTestJSON-125094030-project Aug 07 01:10:12.987197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: Aug 07 01:10:12.987197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:12.987197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '679843ff-5657-404a-a0d0-aafd56316ffc'} Aug 07 01:10:12.987197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. (HTTP 404) (Request-ID: req-c53578cf-9af0-4b77-8597-7ba53461ce16) Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:10:12.987814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 07 01:10:12.988883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 07 01:10:12.989968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:10:12.992926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:12.992926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:12.992926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:10:12.992926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:12.992926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:12.992926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:10:12.992926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:10:12.992926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:10:12.992926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:10:12.992926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:12.992926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:12.995544 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-03732a50-3ec5-4d1e-a190-7076bccc0546 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Returning 404 to user: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:10:12.996835 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-03732a50-3ec5-4d1e-a190-7076bccc0546 tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc" status: 404 len: 109 microversion: 2.1 time: 0.112719 Aug 07 01:10:12.997675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 13/27] 10.222.0.26 () {58 vars in 1248 bytes} [Sun Aug 7 01:10:12 2022] DELETE /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc => generated 109 bytes in 114 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:10:13.004725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9c797712-6de0-41fc-acab-87d1acfc4b5c tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] Calling method '.version_select at 0x7f1fae8ed9d0>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:13.290511 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9c797712-6de0-41fc-acab-87d1acfc4b5c tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] 10.222.0.26 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.291203 Aug 07 01:10:13.291722 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 14/28] 10.222.0.26 () {58 vars in 1134 bytes} [Sun Aug 7 01:10:12 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:13.311819 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-828c34aa-7da4-43ca-ad52-8508d95d88eb tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Calling method '.version_select at 0x7f1fae88b790>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:13.359393 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-828c34aa-7da4-43ca-ad52-8508d95d88eb tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] HTTP exception thrown: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-828c34aa-7da4-43ca-ad52-8508d95d88eb tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Request method failure captured: Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: GET /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc HTTP/1.1 Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xDgs7rHcX-oc9QApK-Hug_aEXBc7reGvuGY3JykitLnQYDia8SOvTO_avJvDbMpNt5wSK_F4H4yl2NbOR6sOTP8UbgR8CIC825kUlEi7ls0399yjBId4cqSwOitPvEITF2_SoJ-lBlrirMhSTXinE_dnhYQ8ZiUR1K8OHQW8S60RGU Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: fce127d62e6648d89d54ab21bd3b14e2 Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-125094030 Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:10:13.362071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:10:13.363795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fce127d62e6648d89d54ab21bd3b14e2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:10:13.363795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-VolumesSnapshotsTestJSON-125094030 Aug 07 01:10:13.363795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: fce127d62e6648d89d54ab21bd3b14e2 Aug 07 01:10:13.363795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-125094030 Aug 07 01:10:13.363795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-VolumesSnapshotsTestJSON-125094030-project Aug 07 01:10:13.363795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:10:13.363795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:10:13.363795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: c59b392a2e1a4f32bba1433afc519d5e Aug 07 01:10:13.363795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-VolumesSnapshotsTestJSON-125094030-project Aug 07 01:10:13.363795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: Aug 07 01:10:13.363795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:13.363795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '679843ff-5657-404a-a0d0-aafd56316ffc'} Aug 07 01:10:13.363795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. (HTTP 404) (Request-ID: req-53818fd8-f4ab-4839-bef2-2ea90d8c35a1) Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:10:13.364572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 07 01:10:13.365579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 07 01:10:13.366518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:10:13.369312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:13.369312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:13.369312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:10:13.369312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:13.369312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:10:13.369312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:10:13.369312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:10:13.369312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:10:13.369312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:10:13.369312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. Aug 07 01:10:13.369312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:10:13.373841 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-828c34aa-7da4-43ca-ad52-8508d95d88eb tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] Returning 404 to user: Volume 679843ff-5657-404a-a0d0-aafd56316ffc could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:10:13.375634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-828c34aa-7da4-43ca-ad52-8508d95d88eb tempest-VolumesSnapshotsTestJSON-125094030 tempest-VolumesSnapshotsTestJSON-125094030-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc" status: 404 len: 109 microversion: 2.1 time: 0.074421 Aug 07 01:10:13.385363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 15/29] 10.222.0.26 () {58 vars in 1245 bytes} [Sun Aug 7 01:10:13 2022] GET /compute/v2.1/os-volumes/679843ff-5657-404a-a0d0-aafd56316ffc => generated 109 bytes in 85 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:10:13.393142 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c0d9f386-5b99-47d7-bb94-51b6d3528c39 tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] Calling method '.version_select at 0x7f1faf238b80>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:13.626249 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c0d9f386-5b99-47d7-bb94-51b6d3528c39 tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] 10.222.0.26 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.239164 Aug 07 01:10:13.627835 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 16/30] 10.222.0.26 () {58 vars in 1167 bytes} [Sun Aug 7 01:10:13 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 353 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:13.650903 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-45ebad88-2973-4679-bf0d-56532838537f tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] Calling method '.version_select at 0x7f1faf19e040>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:13.854558 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8f4de7b9-3d7e-4f96-84c2-cf52b1547469 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Action: 'create', calling method: .version_select at 0x7f1faf109430>, body: {"keypair": {"name": "tempest-keypair-1615386904", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNzITJCugM5WYu5aQ6LQb89fr1A4hn4tfp1ZL1tUrQIwXwJy/aK5lnrEK8FjZ5XIAgqGhaiQDKN+LMiUziyM6oT3uyFYYfVsfPj4AFrhWf5jF//SzEFE3BAbbCLAUW3rjA=="}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:10:13.971692 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-8f4de7b9-3d7e-4f96-84c2-cf52b1547469 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for project 35820b4f79774ac49f43fce915f56d6f. Resources: {'key_pairs'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:10:13.996484 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-45ebad88-2973-4679-bf0d-56532838537f tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.351749 Aug 07 01:10:13.998013 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 17/31] 10.222.0.26 () {58 vars in 1170 bytes} [Sun Aug 7 01:10:13 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:10:14.009404 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-8f4de7b9-3d7e-4f96-84c2-cf52b1547469 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for user e65f0f1ba54f483abff788974b1256cb and project 35820b4f79774ac49f43fce915f56d6f. Resources: {'key_pairs'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:10:14.021784 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a9a52075-17e5-413f-8def-09ab74d12699 tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] Calling method '.version_select at 0x7f1fae89f040>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:14.200978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8f4de7b9-3d7e-4f96-84c2-cf52b1547469 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 1.461067 Aug 07 01:10:14.206746 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 15/32] 10.222.0.26 () {60 vars in 1159 bytes} [Sun Aug 7 01:10:12 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 1469 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:14.250807 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a9a52075-17e5-413f-8def-09ab74d12699 tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.233362 Aug 07 01:10:14.256128 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 18/33] 10.222.0.26 () {58 vars in 1155 bytes} [Sun Aug 7 01:10:14 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:14.292763 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2a3c6e0c-2994-4cb7-b066-c037eaac9e84 tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] Calling method '.version_select at 0x7f1faf0f40d0>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:14.594952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2a3c6e0c-2994-4cb7-b066-c037eaac9e84 tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.318705 Aug 07 01:10:14.595951 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 16/34] 10.222.0.26 () {58 vars in 1188 bytes} [Sun Aug 7 01:10:14 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 353 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:14.620472 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4b895748-fbde-4a5a-a666-9eecdaadec8f tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] Calling method '.version_select at 0x7f1fae8aa280>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:15.026975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4b895748-fbde-4a5a-a666-9eecdaadec8f tempest-VolumesTestJSON-1705272797 tempest-VolumesTestJSON-1705272797-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.411809 Aug 07 01:10:15.029924 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 19/35] 10.222.0.26 () {58 vars in 1155 bytes} [Sun Aug 7 01:10:14 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:10:19.326176 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1512744704", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "96bfca6e-7131-4db0-b142-73ffbffe52dd"}], "security_groups": [{"name": "tempest-securitygroup--1220631868"}], "key_name": "tempest-keypair-7392549", "user_data": "}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:10:20.074993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.network.neutron [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] validate_networks() for [('96bfca6e-7131-4db0-b142-73ffbffe52dd', None, None, None, None, None)] {{(pid=117983) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:10:21.836323 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=4 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:10:21.857012 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=11,nova_api:INSERT=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:10:22.293837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6dcb8b08-2d03-4283-9ef2-5b65e644cd08 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Action: 'create', calling method: .version_select at 0x7f1fae89f1f0>, body: {"keypair": {"name": "tempest-keypair-1012330668", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGBCzVaEmx2H75TiUYW+NUE/u85Gq/L/d8kaoj9htqR5rv2PAonDoiE6kQfO2KH3okL3gKqYnENG8+vqHIvRrkhfz6vaWeecTDsSkTBq8vuZo27jYvb9W5XwOMp2i0RjJQ=="}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:10:22.342312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-6dcb8b08-2d03-4283-9ef2-5b65e644cd08 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Getting quotas for project 7fd9ab6d98a44605a57bd8ae9535a686. Resources: {'key_pairs'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:10:22.358241 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-6dcb8b08-2d03-4283-9ef2-5b65e644cd08 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Getting quotas for user 4ae9c7ef17b746fc8f87e3c22a241b76 and project 7fd9ab6d98a44605a57bd8ae9535a686. Resources: {'key_pairs'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:10:22.446651 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6dcb8b08-2d03-4283-9ef2-5b65e644cd08 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.627617 Aug 07 01:10:22.447613 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 20/36] 10.222.0.26 () {60 vars in 1159 bytes} [Sun Aug 7 01:10:21 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 631 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:23.982374 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Flavor limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:10:23.984225 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Image limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:10:23.985434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Flavor pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:10:23.986438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Image pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:10:23.987565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:10:24.011514 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:24.013668 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:24.014777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:24.016543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:24.017685 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:24.018708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:24.088320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Getting quotas for project 1b1d33b8c71e41e6bf23d4ad8590d2c2. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:10:24.101500 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Getting quotas for user 5fafe42957914b929f9ffee53ab6e7ac and project 1b1d33b8c71e41e6bf23d4ad8590d2c2. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:10:24.160567 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Going to run 1 instances... {{(pid=117983) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:10:24.178571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117983) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:10:24.466963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e5df28ad-c14d-40b4-a3a0-e72a432020e4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 5.172043 Aug 07 01:10:24.480349 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 17/37] 10.222.0.26 () {62 vars in 1188 bytes} [Sun Aug 7 01:10:19 2022] POST /compute/v2.1/servers => generated 402 bytes in 5178 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Aug 07 01:10:24.497526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0ed7d603-5442-4bdf-ab58-f3d21136ac65 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:24.508287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0ed7d603-5442-4bdf-ab58-f3d21136ac65 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:24.927321 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0ed7d603-5442-4bdf-ab58-f3d21136ac65 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:24.970640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0ed7d603-5442-4bdf-ab58-f3d21136ac65 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:24.976506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0ed7d603-5442-4bdf-ab58-f3d21136ac65 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:24.977747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0ed7d603-5442-4bdf-ab58-f3d21136ac65 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1279 microversion: 2.20 time: 0.492821 Aug 07 01:10:24.986627 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 18/38] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:10:24 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1279 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:10:25.088118 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-370110281", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "04909372-4f17-4c44-b384-6e7c31434dac"}], "security_groups": [{"name": "tempest-securitygroup--1345551992"}], "key_name": "tempest-keypair-1615386904", "user_data": "}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:10:25.182300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-685806106", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "networks": [{"uuid": "dd2fdb8b-29a8-4b7e-9b2d-76256c0845cb"}]}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:10:25.686466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.network.neutron [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] validate_networks() for [('dd2fdb8b-29a8-4b7e-9b2d-76256c0845cb', None, None, None, None, None)] {{(pid=117984) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:10:25.753219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.network.neutron [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] validate_networks() for [('04909372-4f17-4c44-b384-6e7c31434dac', None, None, None, None, None)] {{(pid=117983) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:10:28.054975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Flavor limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:10:28.058136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Image limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:10:28.062512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Flavor pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:10:28.066269 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Image pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:10:28.070992 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:10:28.111093 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:28.122933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:28.123394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:28.124260 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:28.127450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:28.127995 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:28.215014 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Getting quotas for project a105c95deb1041ba9ac721d7c82110e9. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:10:28.220960 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Getting quotas for user caafb1ccfd704e3ca65b6c4db04b05ef and project a105c95deb1041ba9ac721d7c82110e9. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:10:28.240568 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Going to run 1 instances... {{(pid=117984) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:10:28.245218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117984) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:10:28.382177 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f361b23d-910f-49b9-97d0-663a28f11c0a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.542800 Aug 07 01:10:28.388804 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 21/39] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:10:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 4558 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 07 01:10:28.416623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-25805a63-91c7-4030-95b9-6c9dba33058e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:28.426910 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-25805a63-91c7-4030-95b9-6c9dba33058e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:28.487742 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-25805a63-91c7-4030-95b9-6c9dba33058e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:28.494596 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-25805a63-91c7-4030-95b9-6c9dba33058e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:28.495731 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-25805a63-91c7-4030-95b9-6c9dba33058e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:28.497878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-25805a63-91c7-4030-95b9-6c9dba33058e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:28.499148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-25805a63-91c7-4030-95b9-6c9dba33058e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:28.500316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-25805a63-91c7-4030-95b9-6c9dba33058e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:28.978535 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-25805a63-91c7-4030-95b9-6c9dba33058e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:29.033187 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-25805a63-91c7-4030-95b9-6c9dba33058e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:29.037644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-25805a63-91c7-4030-95b9-6c9dba33058e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:29.040672 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-25805a63-91c7-4030-95b9-6c9dba33058e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1283 microversion: 2.20 time: 0.648635 Aug 07 01:10:29.055689 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 22/40] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:10:28 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1283 bytes in 666 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:10:29.080345 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-baeb274c-1802-471c-8120-93084a8e643a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:29.083612 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-baeb274c-1802-471c-8120-93084a8e643a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:29.238430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Flavor limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:10:29.240427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Image limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:10:29.241555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Flavor pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:10:29.242687 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Image pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:10:29.243670 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:10:29.263722 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-baeb274c-1802-471c-8120-93084a8e643a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:29.288705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:29.289490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:29.290164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:29.292622 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:29.293503 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:29.293960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:29.315664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-baeb274c-1802-471c-8120-93084a8e643a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1214 microversion: 2.1 time: 0.256570 Aug 07 01:10:29.317317 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 23/41] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:29 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1214 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:29.355972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for project 35820b4f79774ac49f43fce915f56d6f. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:10:29.361794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for user e65f0f1ba54f483abff788974b1256cb and project 35820b4f79774ac49f43fce915f56d6f. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:10:29.393044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Going to run 1 instances... {{(pid=117983) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:10:29.403558 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117983) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:10:29.488645 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5d9edae3-5242-4aa4-944a-4e419779c92a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 4.405624 Aug 07 01:10:29.489965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 19/42] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:10:25 2022] POST /compute/v2.1/servers => generated 402 bytes in 4408 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 07 01:10:29.519714 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-872a9a01-1c12-4b98-b511-bcb775c9ee42 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:29.525695 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-872a9a01-1c12-4b98-b511-bcb775c9ee42 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:30.091536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-97d3af5d-8976-412d-90d7-c9ae80425004 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:30.092644 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-97d3af5d-8976-412d-90d7-c9ae80425004 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:30.109602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97d3af5d-8976-412d-90d7-c9ae80425004 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:30.112753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97d3af5d-8976-412d-90d7-c9ae80425004 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:30.116289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97d3af5d-8976-412d-90d7-c9ae80425004 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:30.130666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97d3af5d-8976-412d-90d7-c9ae80425004 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:30.137452 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97d3af5d-8976-412d-90d7-c9ae80425004 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:30.138282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97d3af5d-8976-412d-90d7-c9ae80425004 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:30.408544 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-872a9a01-1c12-4b98-b511-bcb775c9ee42 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:30.445247 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-872a9a01-1c12-4b98-b511-bcb775c9ee42 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1241 microversion: 2.1 time: 0.935573 Aug 07 01:10:30.471772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 24/43] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:29 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1241 bytes in 964 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:30.506771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-285dd22b-3709-4bb1-bb3f-a574485b8ac7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:30.509507 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-285dd22b-3709-4bb1-bb3f-a574485b8ac7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:30.824856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-285dd22b-3709-4bb1-bb3f-a574485b8ac7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:30.848877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-285dd22b-3709-4bb1-bb3f-a574485b8ac7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1214 microversion: 2.1 time: 0.381057 Aug 07 01:10:30.851528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 25/44] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:30 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1214 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:30.998584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-97d3af5d-8976-412d-90d7-c9ae80425004 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:31.093289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-97d3af5d-8976-412d-90d7-c9ae80425004 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:31.102980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-97d3af5d-8976-412d-90d7-c9ae80425004 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:31.106037 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-97d3af5d-8976-412d-90d7-c9ae80425004 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1275 microversion: 2.20 time: 1.021166 Aug 07 01:10:31.125581 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1404058805", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "b1257e6a-7a97-43f5-969f-654d75e2ab7d"}], "security_groups": [{"name": "tempest-securitygroup--350427165"}], "key_name": "tempest-keypair-1012330668", "user_data": "}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:10:31.134730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 20/45] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:10:30 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1275 bytes in 1051 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:10:31.501950 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-78d3faf2-621c-421d-851d-8250cc960d1f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:31.508543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-78d3faf2-621c-421d-851d-8250cc960d1f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:31.550231 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78d3faf2-621c-421d-851d-8250cc960d1f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:31.554389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78d3faf2-621c-421d-851d-8250cc960d1f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:31.559472 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78d3faf2-621c-421d-851d-8250cc960d1f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:31.563663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78d3faf2-621c-421d-851d-8250cc960d1f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:31.568612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78d3faf2-621c-421d-851d-8250cc960d1f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:31.573211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78d3faf2-621c-421d-851d-8250cc960d1f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:31.656725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.network.neutron [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] validate_networks() for [('b1257e6a-7a97-43f5-969f-654d75e2ab7d', None, None, None, None, None)] {{(pid=117984) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:10:31.928397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-78d3faf2-621c-421d-851d-8250cc960d1f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:31.962207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-78d3faf2-621c-421d-851d-8250cc960d1f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1245 microversion: 2.1 time: 0.473196 Aug 07 01:10:31.976675 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 21/46] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:31 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1245 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:32.012490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-af8afd80-4df1-45d2-81ce-bc458f91e95a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:32.014041 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-af8afd80-4df1-45d2-81ce-bc458f91e95a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:32.028717 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af8afd80-4df1-45d2-81ce-bc458f91e95a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:32.030335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af8afd80-4df1-45d2-81ce-bc458f91e95a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:32.031343 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af8afd80-4df1-45d2-81ce-bc458f91e95a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:32.033251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af8afd80-4df1-45d2-81ce-bc458f91e95a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:32.034220 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af8afd80-4df1-45d2-81ce-bc458f91e95a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:32.035977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af8afd80-4df1-45d2-81ce-bc458f91e95a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:32.672662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-af8afd80-4df1-45d2-81ce-bc458f91e95a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:32.728761 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-af8afd80-4df1-45d2-81ce-bc458f91e95a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1218 microversion: 2.1 time: 0.735685 Aug 07 01:10:32.732540 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 22/47] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:31 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1218 bytes in 755 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:32.756494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-eec24f65-4c4d-4232-965a-c24483d6dbc5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:32.759720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-eec24f65-4c4d-4232-965a-c24483d6dbc5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:32.780009 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eec24f65-4c4d-4232-965a-c24483d6dbc5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:32.781376 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eec24f65-4c4d-4232-965a-c24483d6dbc5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:32.784112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eec24f65-4c4d-4232-965a-c24483d6dbc5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:32.785664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eec24f65-4c4d-4232-965a-c24483d6dbc5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:32.786920 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eec24f65-4c4d-4232-965a-c24483d6dbc5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:32.787906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eec24f65-4c4d-4232-965a-c24483d6dbc5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:33.015227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-eec24f65-4c4d-4232-965a-c24483d6dbc5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:33.083551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-eec24f65-4c4d-4232-965a-c24483d6dbc5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:33.094305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-eec24f65-4c4d-4232-965a-c24483d6dbc5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:33.102298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-eec24f65-4c4d-4232-965a-c24483d6dbc5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1337 microversion: 2.20 time: 0.365102 Aug 07 01:10:33.107265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 23/48] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:10:32 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1337 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:10:33.138030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2bc67c1a-c60c-4ffe-9d3d-c12864a89169 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:33.147122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2bc67c1a-c60c-4ffe-9d3d-c12864a89169 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:33.220013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2bc67c1a-c60c-4ffe-9d3d-c12864a89169 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:33.230582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2bc67c1a-c60c-4ffe-9d3d-c12864a89169 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:33.232219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2bc67c1a-c60c-4ffe-9d3d-c12864a89169 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:33.235737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2bc67c1a-c60c-4ffe-9d3d-c12864a89169 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:33.252888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2bc67c1a-c60c-4ffe-9d3d-c12864a89169 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:33.257107 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2bc67c1a-c60c-4ffe-9d3d-c12864a89169 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:33.513674 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Flavor limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:10:33.514725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Image limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:10:33.515684 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Flavor pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:10:33.517179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Image pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:10:33.518051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:10:33.596606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:33.601976 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:33.603947 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:33.610419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:33.618503 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:33.634564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:33.741527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Getting quotas for project 7fd9ab6d98a44605a57bd8ae9535a686. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:10:33.760524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Getting quotas for user 4ae9c7ef17b746fc8f87e3c22a241b76 and project 7fd9ab6d98a44605a57bd8ae9535a686. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:10:33.799332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Going to run 1 instances... {{(pid=117984) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:10:33.811465 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117984) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:10:33.970522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a0d4b6ef-2d53-4e3e-acea-b52267b55f2e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 2.860156 Aug 07 01:10:33.984301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 26/49] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:10:31 2022] POST /compute/v2.1/servers => generated 401 bytes in 2875 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:10:34.016526 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-db1934e3-7c2c-4907-956b-4956aaff52a0 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:34.016526 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-db1934e3-7c2c-4907-956b-4956aaff52a0 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:34.064655 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-db1934e3-7c2c-4907-956b-4956aaff52a0 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:34.068929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-db1934e3-7c2c-4907-956b-4956aaff52a0 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:34.071579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-db1934e3-7c2c-4907-956b-4956aaff52a0 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:34.073105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-db1934e3-7c2c-4907-956b-4956aaff52a0 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:34.074254 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-db1934e3-7c2c-4907-956b-4956aaff52a0 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:34.077710 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-db1934e3-7c2c-4907-956b-4956aaff52a0 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:34.090598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2bc67c1a-c60c-4ffe-9d3d-c12864a89169 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:34.129657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2bc67c1a-c60c-4ffe-9d3d-c12864a89169 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1301 microversion: 2.1 time: 1.015893 Aug 07 01:10:34.150651 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 24/50] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:33 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1301 bytes in 1041 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:34.210191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b6fce645-ab13-4640-9c3e-012efdc2bfac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:34.211362 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b6fce645-ab13-4640-9c3e-012efdc2bfac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:34.488538 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-db1934e3-7c2c-4907-956b-4956aaff52a0 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:34.537810 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-db1934e3-7c2c-4907-956b-4956aaff52a0 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1284 microversion: 2.1 time: 0.550848 Aug 07 01:10:34.548985 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 27/51] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:33 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1284 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:34.568252 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e0256485-1a8c-44df-8698-7ab638673599 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:34.587235 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e0256485-1a8c-44df-8698-7ab638673599 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:34.638077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0256485-1a8c-44df-8698-7ab638673599 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:34.638822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0256485-1a8c-44df-8698-7ab638673599 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:34.639589 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0256485-1a8c-44df-8698-7ab638673599 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:34.649712 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0256485-1a8c-44df-8698-7ab638673599 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:34.652315 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0256485-1a8c-44df-8698-7ab638673599 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:34.653085 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0256485-1a8c-44df-8698-7ab638673599 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:34.661309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b6fce645-ab13-4640-9c3e-012efdc2bfac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:34.695138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b6fce645-ab13-4640-9c3e-012efdc2bfac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1238 microversion: 2.1 time: 0.542246 Aug 07 01:10:34.698314 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 25/52] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:34 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1238 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:35.026164 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e0256485-1a8c-44df-8698-7ab638673599 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:35.045485 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e0256485-1a8c-44df-8698-7ab638673599 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:35.054047 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e0256485-1a8c-44df-8698-7ab638673599 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:35.061018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e0256485-1a8c-44df-8698-7ab638673599 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1337 microversion: 2.20 time: 0.504845 Aug 07 01:10:35.061842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 28/53] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:10:34 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1337 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:10:35.189868 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f63c0b51-dee5-47a5-a120-633b13a6b81f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:35.196225 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f63c0b51-dee5-47a5-a120-633b13a6b81f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:35.227637 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f63c0b51-dee5-47a5-a120-633b13a6b81f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:35.233248 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f63c0b51-dee5-47a5-a120-633b13a6b81f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:35.236561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f63c0b51-dee5-47a5-a120-633b13a6b81f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:35.240809 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f63c0b51-dee5-47a5-a120-633b13a6b81f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:35.259517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f63c0b51-dee5-47a5-a120-633b13a6b81f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:35.260375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f63c0b51-dee5-47a5-a120-633b13a6b81f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:35.606403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-649d32ac-2c79-476f-a26b-f1c95f82c1d5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:35.617168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-649d32ac-2c79-476f-a26b-f1c95f82c1d5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:35.685387 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-649d32ac-2c79-476f-a26b-f1c95f82c1d5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:35.691916 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-649d32ac-2c79-476f-a26b-f1c95f82c1d5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:35.696950 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-649d32ac-2c79-476f-a26b-f1c95f82c1d5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:35.702728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-649d32ac-2c79-476f-a26b-f1c95f82c1d5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:35.710200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-649d32ac-2c79-476f-a26b-f1c95f82c1d5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:35.711657 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-649d32ac-2c79-476f-a26b-f1c95f82c1d5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:35.964811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f63c0b51-dee5-47a5-a120-633b13a6b81f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:35.992167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f63c0b51-dee5-47a5-a120-633b13a6b81f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1299 microversion: 2.1 time: 0.809119 Aug 07 01:10:35.997488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 26/54] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:35 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1299 bytes in 819 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:36.048522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a9816bcb-331c-4f23-be07-6167cc5ee3bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:36.050044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a9816bcb-331c-4f23-be07-6167cc5ee3bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:36.053137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-649d32ac-2c79-476f-a26b-f1c95f82c1d5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:36.102552 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9816bcb-331c-4f23-be07-6167cc5ee3bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:36.106261 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-649d32ac-2c79-476f-a26b-f1c95f82c1d5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1272 microversion: 2.1 time: 0.517685 Aug 07 01:10:36.112508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9816bcb-331c-4f23-be07-6167cc5ee3bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:36.117945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 29/55] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:35 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1272 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:36.118978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9816bcb-331c-4f23-be07-6167cc5ee3bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:36.122732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9816bcb-331c-4f23-be07-6167cc5ee3bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:36.129195 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9816bcb-331c-4f23-be07-6167cc5ee3bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:36.129770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9816bcb-331c-4f23-be07-6167cc5ee3bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:36.142429 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-211ebdd4-c01e-4b47-9864-9753c19b4580 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:36.143657 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-211ebdd4-c01e-4b47-9864-9753c19b4580 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:36.189695 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-211ebdd4-c01e-4b47-9864-9753c19b4580 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:36.190661 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-211ebdd4-c01e-4b47-9864-9753c19b4580 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:36.191742 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-211ebdd4-c01e-4b47-9864-9753c19b4580 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:36.192717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-211ebdd4-c01e-4b47-9864-9753c19b4580 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:36.193342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-211ebdd4-c01e-4b47-9864-9753c19b4580 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:36.193725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-211ebdd4-c01e-4b47-9864-9753c19b4580 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:36.816980 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-211ebdd4-c01e-4b47-9864-9753c19b4580 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:36.839820 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-211ebdd4-c01e-4b47-9864-9753c19b4580 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:36.846791 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-211ebdd4-c01e-4b47-9864-9753c19b4580 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:36.855839 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-211ebdd4-c01e-4b47-9864-9753c19b4580 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1337 microversion: 2.20 time: 0.735389 Aug 07 01:10:36.859239 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 30/56] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:10:36 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1337 bytes in 741 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:10:37.052740 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3d1b381a-e467-49fc-9762-6c1836dbae7d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:37.060530 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3d1b381a-e467-49fc-9762-6c1836dbae7d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:37.123862 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3d1b381a-e467-49fc-9762-6c1836dbae7d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:37.127231 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3d1b381a-e467-49fc-9762-6c1836dbae7d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:37.130233 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3d1b381a-e467-49fc-9762-6c1836dbae7d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:37.133221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3d1b381a-e467-49fc-9762-6c1836dbae7d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:37.134964 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3d1b381a-e467-49fc-9762-6c1836dbae7d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:37.146746 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3d1b381a-e467-49fc-9762-6c1836dbae7d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:37.957766 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3d1b381a-e467-49fc-9762-6c1836dbae7d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:38.020723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3d1b381a-e467-49fc-9762-6c1836dbae7d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1299 microversion: 2.1 time: 0.979435 Aug 07 01:10:38.025224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 31/57] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:37 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1299 bytes in 994 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:10:38.047642 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5d400f32-7dda-4074-9de4-89bfac0b8fff tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:38.074725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5d400f32-7dda-4074-9de4-89bfac0b8fff tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:38.118893 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d400f32-7dda-4074-9de4-89bfac0b8fff tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:38.133481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d400f32-7dda-4074-9de4-89bfac0b8fff tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:38.136279 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d400f32-7dda-4074-9de4-89bfac0b8fff tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:38.138298 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d400f32-7dda-4074-9de4-89bfac0b8fff tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:38.148064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d400f32-7dda-4074-9de4-89bfac0b8fff tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:38.157643 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d400f32-7dda-4074-9de4-89bfac0b8fff tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:38.343159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a9816bcb-331c-4f23-be07-6167cc5ee3bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:38.413611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a9816bcb-331c-4f23-be07-6167cc5ee3bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1234 microversion: 2.1 time: 2.414112 Aug 07 01:10:38.443020 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 27/58] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:35 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1234 bytes in 2445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:38.461315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-476a12ff-fad4-4017-809a-1a1e522e5a28 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:38.489303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-476a12ff-fad4-4017-809a-1a1e522e5a28 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:38.534195 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-476a12ff-fad4-4017-809a-1a1e522e5a28 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:38.559295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-476a12ff-fad4-4017-809a-1a1e522e5a28 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.024s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:38.559900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-476a12ff-fad4-4017-809a-1a1e522e5a28 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:38.565795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-476a12ff-fad4-4017-809a-1a1e522e5a28 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:38.582242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-476a12ff-fad4-4017-809a-1a1e522e5a28 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:38.583097 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-476a12ff-fad4-4017-809a-1a1e522e5a28 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:39.314080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=4 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:10:41.294681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-476a12ff-fad4-4017-809a-1a1e522e5a28 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:41.342457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-476a12ff-fad4-4017-809a-1a1e522e5a28 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:41.405435 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-476a12ff-fad4-4017-809a-1a1e522e5a28 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:41.412499 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-476a12ff-fad4-4017-809a-1a1e522e5a28 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1405 microversion: 2.20 time: 2.959613 Aug 07 01:10:41.459931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 28/59] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:10:38 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1405 bytes in 3016 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:10:41.498488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-83ee08d8-b192-40b3-b91a-5c447f7f4f3b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:41.510548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-83ee08d8-b192-40b3-b91a-5c447f7f4f3b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:41.578137 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83ee08d8-b192-40b3-b91a-5c447f7f4f3b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:41.605391 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83ee08d8-b192-40b3-b91a-5c447f7f4f3b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:41.606356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83ee08d8-b192-40b3-b91a-5c447f7f4f3b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.027s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:41.612117 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83ee08d8-b192-40b3-b91a-5c447f7f4f3b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:41.634899 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83ee08d8-b192-40b3-b91a-5c447f7f4f3b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.023s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:41.648878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83ee08d8-b192-40b3-b91a-5c447f7f4f3b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:42.897109 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5d400f32-7dda-4074-9de4-89bfac0b8fff tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:42.924192 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5d400f32-7dda-4074-9de4-89bfac0b8fff tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1314 microversion: 2.1 time: 4.896421 Aug 07 01:10:42.925306 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 32/60] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:38 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1314 bytes in 4899 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:42.965439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-caeb68cc-551a-4b4b-a775-c5ecd4180952 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:42.966585 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-caeb68cc-551a-4b4b-a775-c5ecd4180952 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:42.999293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-caeb68cc-551a-4b4b-a775-c5ecd4180952 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:43.003150 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-caeb68cc-551a-4b4b-a775-c5ecd4180952 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:43.005167 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-caeb68cc-551a-4b4b-a775-c5ecd4180952 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:43.006527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-caeb68cc-551a-4b4b-a775-c5ecd4180952 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:43.009361 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-caeb68cc-551a-4b4b-a775-c5ecd4180952 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:43.013444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-caeb68cc-551a-4b4b-a775-c5ecd4180952 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:43.303194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-83ee08d8-b192-40b3-b91a-5c447f7f4f3b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:43.305432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-caeb68cc-551a-4b4b-a775-c5ecd4180952 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:43.320852 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-83ee08d8-b192-40b3-b91a-5c447f7f4f3b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1367 microversion: 2.1 time: 1.854722 Aug 07 01:10:43.321982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 29/61] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:41 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1367 bytes in 1861 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:43.338233 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8b40d70f-74d3-4c01-bbcc-5e772ffccec8 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:43.339639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8b40d70f-74d3-4c01-bbcc-5e772ffccec8 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:43.368091 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-caeb68cc-551a-4b4b-a775-c5ecd4180952 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1296 microversion: 2.1 time: 0.425161 Aug 07 01:10:43.373924 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 33/62] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:42 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1296 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:43.376797 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b40d70f-74d3-4c01-bbcc-5e772ffccec8 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:43.389341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b40d70f-74d3-4c01-bbcc-5e772ffccec8 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:43.396582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b40d70f-74d3-4c01-bbcc-5e772ffccec8 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:43.401042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b40d70f-74d3-4c01-bbcc-5e772ffccec8 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:43.411442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b40d70f-74d3-4c01-bbcc-5e772ffccec8 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:43.414269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b40d70f-74d3-4c01-bbcc-5e772ffccec8 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:43.977135 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-558d626a-d2b4-4bb6-8670-b74bf987a425 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:43.984786 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-558d626a-d2b4-4bb6-8670-b74bf987a425 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:44.028730 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=4 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:10:44.033686 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=22 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:10:44.041919 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-558d626a-d2b4-4bb6-8670-b74bf987a425 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:44.045138 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-558d626a-d2b4-4bb6-8670-b74bf987a425 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:44.047076 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-558d626a-d2b4-4bb6-8670-b74bf987a425 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:44.049105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-558d626a-d2b4-4bb6-8670-b74bf987a425 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:44.050557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-558d626a-d2b4-4bb6-8670-b74bf987a425 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:44.051531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-558d626a-d2b4-4bb6-8670-b74bf987a425 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:44.689977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=20 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:10:44.820897 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-558d626a-d2b4-4bb6-8670-b74bf987a425 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:44.841144 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-558d626a-d2b4-4bb6-8670-b74bf987a425 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1314 microversion: 2.1 time: 0.882360 Aug 07 01:10:44.844588 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 34/63] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:43 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1314 bytes in 888 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:10:44.872779 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3b44669f-c48f-4cc1-9e5e-ee7243d5063e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:44.875103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3b44669f-c48f-4cc1-9e5e-ee7243d5063e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:44.896667 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b44669f-c48f-4cc1-9e5e-ee7243d5063e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:44.901714 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b44669f-c48f-4cc1-9e5e-ee7243d5063e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:44.902987 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b44669f-c48f-4cc1-9e5e-ee7243d5063e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:44.905978 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b44669f-c48f-4cc1-9e5e-ee7243d5063e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:44.909467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b44669f-c48f-4cc1-9e5e-ee7243d5063e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:44.913036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b44669f-c48f-4cc1-9e5e-ee7243d5063e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:45.461757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8b40d70f-74d3-4c01-bbcc-5e772ffccec8 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:45.521871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8b40d70f-74d3-4c01-bbcc-5e772ffccec8 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:45.529601 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8b40d70f-74d3-4c01-bbcc-5e772ffccec8 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:45.543922 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8b40d70f-74d3-4c01-bbcc-5e772ffccec8 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1405 microversion: 2.20 time: 2.220052 Aug 07 01:10:45.547036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 30/64] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:10:43 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1405 bytes in 2224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:10:45.596961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a504341c-5bc5-4ffc-bbfe-b4bed88e684c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:45.598379 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a504341c-5bc5-4ffc-bbfe-b4bed88e684c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:45.616604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=70,nova_api:INSERT=7 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:10:45.619382 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a504341c-5bc5-4ffc-bbfe-b4bed88e684c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:45.624848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a504341c-5bc5-4ffc-bbfe-b4bed88e684c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:45.626264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a504341c-5bc5-4ffc-bbfe-b4bed88e684c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:45.630389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a504341c-5bc5-4ffc-bbfe-b4bed88e684c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:45.634924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a504341c-5bc5-4ffc-bbfe-b4bed88e684c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:45.636049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a504341c-5bc5-4ffc-bbfe-b4bed88e684c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:45.929413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3b44669f-c48f-4cc1-9e5e-ee7243d5063e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:45.944279 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3b44669f-c48f-4cc1-9e5e-ee7243d5063e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1367 microversion: 2.1 time: 1.090748 Aug 07 01:10:45.946718 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 35/65] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:44 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1367 bytes in 1098 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:45.951982 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-698b85ba-dd1a-4002-a63a-4621d6cd766b tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:45.952677 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-698b85ba-dd1a-4002-a63a-4621d6cd766b tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:45.987392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-698b85ba-dd1a-4002-a63a-4621d6cd766b tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:45.989071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-698b85ba-dd1a-4002-a63a-4621d6cd766b tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:45.990311 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-698b85ba-dd1a-4002-a63a-4621d6cd766b tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:45.997533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-698b85ba-dd1a-4002-a63a-4621d6cd766b tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:45.998092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-698b85ba-dd1a-4002-a63a-4621d6cd766b tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:46.007302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-698b85ba-dd1a-4002-a63a-4621d6cd766b tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:46.255473 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a504341c-5bc5-4ffc-bbfe-b4bed88e684c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:46.283060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a504341c-5bc5-4ffc-bbfe-b4bed88e684c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1363 microversion: 2.1 time: 0.726735 Aug 07 01:10:46.283996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 31/66] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:45 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1363 bytes in 736 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:46.479098 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-698b85ba-dd1a-4002-a63a-4621d6cd766b tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:46.516115 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-698b85ba-dd1a-4002-a63a-4621d6cd766b tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1314 microversion: 2.1 time: 0.567822 Aug 07 01:10:46.517164 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 36/67] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:45 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1314 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:46.578872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-992be841-8492-4cda-9bc0-74f9d8afb909 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:46.580558 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-992be841-8492-4cda-9bc0-74f9d8afb909 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:46.592872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-992be841-8492-4cda-9bc0-74f9d8afb909 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:46.594078 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-992be841-8492-4cda-9bc0-74f9d8afb909 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:46.595123 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-992be841-8492-4cda-9bc0-74f9d8afb909 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:46.598146 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-992be841-8492-4cda-9bc0-74f9d8afb909 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:46.599275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-992be841-8492-4cda-9bc0-74f9d8afb909 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:46.600324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-992be841-8492-4cda-9bc0-74f9d8afb909 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:46.763647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-f648a9d3-36d4-450a-be39-4cadf21534a1 req-964df7df-7302-43fb-a6ea-f3604689eda9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7f60fb22-d17a-4b7c-9b7d-95543be1907e", "tag": "9ab59ce3-ed0d-4a8e-b76b-adeff559725a"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:10:46.789136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-f648a9d3-36d4-450a-be39-4cadf21534a1 req-964df7df-7302-43fb-a6ea-f3604689eda9 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:46.791499 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-f648a9d3-36d4-450a-be39-4cadf21534a1 req-964df7df-7302-43fb-a6ea-f3604689eda9 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:46.793466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-f648a9d3-36d4-450a-be39-4cadf21534a1 req-964df7df-7302-43fb-a6ea-f3604689eda9 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:46.849733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-f648a9d3-36d4-450a-be39-4cadf21534a1 req-964df7df-7302-43fb-a6ea-f3604689eda9 service nova] Creating event network-changed:9ab59ce3-ed0d-4a8e-b76b-adeff559725a for instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e on hv-cinder-maste Aug 07 01:10:46.850737 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-f648a9d3-36d4-450a-be39-4cadf21534a1 req-964df7df-7302-43fb-a6ea-f3604689eda9 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=117984) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:293}} Aug 07 01:10:46.851434 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-f648a9d3-36d4-450a-be39-4cadf21534a1 req-964df7df-7302-43fb-a6ea-f3604689eda9 service nova] Acquired lock "compute-rpcapi-router" {{(pid=117984) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:296}} Aug 07 01:10:46.853390 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd495510-b02c-40d1-9210-86f7b47bddb2 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:46.854202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd495510-b02c-40d1-9210-86f7b47bddb2 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:46.855216 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd495510-b02c-40d1-9210-86f7b47bddb2 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:46.856867 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd495510-b02c-40d1-9210-86f7b47bddb2 None None] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:46.858028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd495510-b02c-40d1-9210-86f7b47bddb2 None None] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:46.858755 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd495510-b02c-40d1-9210-86f7b47bddb2 None None] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:46.877207 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.compute.rpcapi [None req-fd495510-b02c-40d1-9210-86f7b47bddb2 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Aug 07 01:10:46.879340 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd495510-b02c-40d1-9210-86f7b47bddb2 None None] Releasing lock "compute-rpcapi-router" {{(pid=117984) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:314}} Aug 07 01:10:46.914540 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fd495510-b02c-40d1-9210-86f7b47bddb2 None None] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.256164 Aug 07 01:10:46.916178 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 37/68] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:10:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:46.993497 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fb1c2a32-ada3-4ead-aa98-e818564bb63f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:47.001487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fb1c2a32-ada3-4ead-aa98-e818564bb63f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:47.025618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb1c2a32-ada3-4ead-aa98-e818564bb63f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:47.036535 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb1c2a32-ada3-4ead-aa98-e818564bb63f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:47.037845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb1c2a32-ada3-4ead-aa98-e818564bb63f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:47.039534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb1c2a32-ada3-4ead-aa98-e818564bb63f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:47.041658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb1c2a32-ada3-4ead-aa98-e818564bb63f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:47.045888 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb1c2a32-ada3-4ead-aa98-e818564bb63f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:47.249745 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-992be841-8492-4cda-9bc0-74f9d8afb909 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:47.270637 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-992be841-8492-4cda-9bc0-74f9d8afb909 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:47.290069 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-992be841-8492-4cda-9bc0-74f9d8afb909 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:47.294293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-992be841-8492-4cda-9bc0-74f9d8afb909 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1405 microversion: 2.20 time: 0.720133 Aug 07 01:10:47.298299 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 32/69] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:10:46 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1405 bytes in 728 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:10:47.472958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fb1c2a32-ada3-4ead-aa98-e818564bb63f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:47.488475 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fb1c2a32-ada3-4ead-aa98-e818564bb63f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1367 microversion: 2.1 time: 0.506090 Aug 07 01:10:47.489830 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 38/70] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:46 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1367 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:10:47.495492 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-07d46324-29d9-4137-8a9b-87ace5e8fdfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:47.496456 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-07d46324-29d9-4137-8a9b-87ace5e8fdfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:47.516953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07d46324-29d9-4137-8a9b-87ace5e8fdfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:47.525131 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07d46324-29d9-4137-8a9b-87ace5e8fdfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:47.525961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07d46324-29d9-4137-8a9b-87ace5e8fdfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:47.526934 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07d46324-29d9-4137-8a9b-87ace5e8fdfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:47.527570 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07d46324-29d9-4137-8a9b-87ace5e8fdfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:47.528191 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07d46324-29d9-4137-8a9b-87ace5e8fdfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:47.579565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-ee65df44-5e5d-441d-81c8-2600660c8e87 req-c36ac7cd-934d-4a1c-a063-8dddeedf7a9d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "93936488-825c-4db2-b648-ac856631c434", "tag": "a2435539-4100-4730-a912-3afae7d39093"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:10:47.685284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-ee65df44-5e5d-441d-81c8-2600660c8e87 req-c36ac7cd-934d-4a1c-a063-8dddeedf7a9d service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:47.686280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-ee65df44-5e5d-441d-81c8-2600660c8e87 req-c36ac7cd-934d-4a1c-a063-8dddeedf7a9d service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:47.686766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-ee65df44-5e5d-441d-81c8-2600660c8e87 req-c36ac7cd-934d-4a1c-a063-8dddeedf7a9d service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:47.792368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-ee65df44-5e5d-441d-81c8-2600660c8e87 req-c36ac7cd-934d-4a1c-a063-8dddeedf7a9d service nova] Creating event network-changed:a2435539-4100-4730-a912-3afae7d39093 for instance 93936488-825c-4db2-b648-ac856631c434 on hv-cinder-maste Aug 07 01:10:47.805958 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-ee65df44-5e5d-441d-81c8-2600660c8e87 req-c36ac7cd-934d-4a1c-a063-8dddeedf7a9d service nova] Acquiring lock "compute-rpcapi-router" {{(pid=117983) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:293}} Aug 07 01:10:47.806410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-ee65df44-5e5d-441d-81c8-2600660c8e87 req-c36ac7cd-934d-4a1c-a063-8dddeedf7a9d service nova] Acquired lock "compute-rpcapi-router" {{(pid=117983) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:296}} Aug 07 01:10:47.814647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-624bbc00-890c-4b7a-919d-4059790c57cf None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:47.815231 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-624bbc00-890c-4b7a-919d-4059790c57cf None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:47.816018 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-624bbc00-890c-4b7a-919d-4059790c57cf None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:47.821167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-624bbc00-890c-4b7a-919d-4059790c57cf None None] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:47.829128 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-624bbc00-890c-4b7a-919d-4059790c57cf None None] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:47.832930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-624bbc00-890c-4b7a-919d-4059790c57cf None None] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:47.886754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.compute.rpcapi [None req-624bbc00-890c-4b7a-919d-4059790c57cf None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Aug 07 01:10:47.894360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-624bbc00-890c-4b7a-919d-4059790c57cf None None] Releasing lock "compute-rpcapi-router" {{(pid=117983) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:314}} Aug 07 01:10:47.977273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-624bbc00-890c-4b7a-919d-4059790c57cf None None] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.676188 Aug 07 01:10:47.978299 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 33/71] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:10:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 679 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:47.989319 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-204c76e2-565f-44fa-9b1a-2b8d92c541f5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:47.992357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-204c76e2-565f-44fa-9b1a-2b8d92c541f5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:48.007535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-204c76e2-565f-44fa-9b1a-2b8d92c541f5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:48.015994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-204c76e2-565f-44fa-9b1a-2b8d92c541f5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:48.021444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-204c76e2-565f-44fa-9b1a-2b8d92c541f5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:48.023163 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-204c76e2-565f-44fa-9b1a-2b8d92c541f5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:48.034038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-204c76e2-565f-44fa-9b1a-2b8d92c541f5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:48.035566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-204c76e2-565f-44fa-9b1a-2b8d92c541f5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:48.885822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-07d46324-29d9-4137-8a9b-87ace5e8fdfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:48.909873 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-07d46324-29d9-4137-8a9b-87ace5e8fdfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1363 microversion: 2.1 time: 1.418420 Aug 07 01:10:48.935011 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 39/72] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:47 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1363 bytes in 1444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:48.940703 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-478c4b66-adc4-48f5-9746-c35196c5df1b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:48.942456 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-478c4b66-adc4-48f5-9746-c35196c5df1b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:48.950419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-478c4b66-adc4-48f5-9746-c35196c5df1b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:48.951529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-478c4b66-adc4-48f5-9746-c35196c5df1b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:48.952214 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-478c4b66-adc4-48f5-9746-c35196c5df1b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:48.953232 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-478c4b66-adc4-48f5-9746-c35196c5df1b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:48.953987 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-478c4b66-adc4-48f5-9746-c35196c5df1b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:48.954627 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-478c4b66-adc4-48f5-9746-c35196c5df1b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:49.395752 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-204c76e2-565f-44fa-9b1a-2b8d92c541f5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:49.416936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-204c76e2-565f-44fa-9b1a-2b8d92c541f5 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1314 microversion: 2.1 time: 1.436588 Aug 07 01:10:49.418095 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 34/73] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:47 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1314 bytes in 1439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:49.455016 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0499cc11-ccad-4aff-a344-f5193349a191 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:49.456028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0499cc11-ccad-4aff-a344-f5193349a191 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:49.477949 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0499cc11-ccad-4aff-a344-f5193349a191 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:49.489841 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0499cc11-ccad-4aff-a344-f5193349a191 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:49.491067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0499cc11-ccad-4aff-a344-f5193349a191 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:49.492688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0499cc11-ccad-4aff-a344-f5193349a191 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:49.493718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0499cc11-ccad-4aff-a344-f5193349a191 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:49.494872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0499cc11-ccad-4aff-a344-f5193349a191 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:51.171208 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-478c4b66-adc4-48f5-9746-c35196c5df1b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:51.187629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-478c4b66-adc4-48f5-9746-c35196c5df1b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:51.200603 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-478c4b66-adc4-48f5-9746-c35196c5df1b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:51.202994 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-478c4b66-adc4-48f5-9746-c35196c5df1b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1405 microversion: 2.20 time: 2.266144 Aug 07 01:10:51.209105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 40/74] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:10:48 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1405 bytes in 2273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:10:51.230161 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-c2c61eb2-b0e9-44b2-b789-406446376038 req-8eb825cf-d12e-422b-9d29-8be993d949e1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "96f9a9ce-f29e-48be-a36b-8518126a09ee", "tag": "f71ffe87-943a-40bf-94f2-b17532b7092c"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:10:51.281258 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c2c61eb2-b0e9-44b2-b789-406446376038 req-8eb825cf-d12e-422b-9d29-8be993d949e1 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:51.283338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c2c61eb2-b0e9-44b2-b789-406446376038 req-8eb825cf-d12e-422b-9d29-8be993d949e1 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:51.289230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c2c61eb2-b0e9-44b2-b789-406446376038 req-8eb825cf-d12e-422b-9d29-8be993d949e1 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:51.367069 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-c2c61eb2-b0e9-44b2-b789-406446376038 req-8eb825cf-d12e-422b-9d29-8be993d949e1 service nova] Creating event network-changed:f71ffe87-943a-40bf-94f2-b17532b7092c for instance 96f9a9ce-f29e-48be-a36b-8518126a09ee on hv-cinder-maste Aug 07 01:10:51.380822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-c2c61eb2-b0e9-44b2-b789-406446376038 req-8eb825cf-d12e-422b-9d29-8be993d949e1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.165901 Aug 07 01:10:51.382030 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 41/75] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:10:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:51.407914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-60c4d8d4-0fb6-4726-bde1-eab3228327c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:51.414659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-60c4d8d4-0fb6-4726-bde1-eab3228327c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:51.478268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60c4d8d4-0fb6-4726-bde1-eab3228327c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:51.484850 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60c4d8d4-0fb6-4726-bde1-eab3228327c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:51.493582 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60c4d8d4-0fb6-4726-bde1-eab3228327c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:51.503639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60c4d8d4-0fb6-4726-bde1-eab3228327c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:51.510776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60c4d8d4-0fb6-4726-bde1-eab3228327c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:51.512840 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60c4d8d4-0fb6-4726-bde1-eab3228327c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:51.567617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0499cc11-ccad-4aff-a344-f5193349a191 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:51.612126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0499cc11-ccad-4aff-a344-f5193349a191 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1367 microversion: 2.1 time: 2.188942 Aug 07 01:10:51.616226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 35/76] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:49 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1367 bytes in 2198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:51.644988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-95dd6893-ecf6-4545-a3ff-0f24bf3d0749 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:51.646887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-95dd6893-ecf6-4545-a3ff-0f24bf3d0749 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:51.693849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95dd6893-ecf6-4545-a3ff-0f24bf3d0749 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:51.706150 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95dd6893-ecf6-4545-a3ff-0f24bf3d0749 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:51.707980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95dd6893-ecf6-4545-a3ff-0f24bf3d0749 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:51.711075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95dd6893-ecf6-4545-a3ff-0f24bf3d0749 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:51.717590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95dd6893-ecf6-4545-a3ff-0f24bf3d0749 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:51.729893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95dd6893-ecf6-4545-a3ff-0f24bf3d0749 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:52.777944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-95dd6893-ecf6-4545-a3ff-0f24bf3d0749 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:52.802771 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-95dd6893-ecf6-4545-a3ff-0f24bf3d0749 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1314 microversion: 2.1 time: 1.184886 Aug 07 01:10:52.804114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 36/77] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:51 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1314 bytes in 1187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:52.811970 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-12538484-3b8a-45ca-bde2-93991ccfaa0d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:52.812984 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-12538484-3b8a-45ca-bde2-93991ccfaa0d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:52.834339 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-12538484-3b8a-45ca-bde2-93991ccfaa0d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:52.835768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-12538484-3b8a-45ca-bde2-93991ccfaa0d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:52.836822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-12538484-3b8a-45ca-bde2-93991ccfaa0d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:52.838593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-12538484-3b8a-45ca-bde2-93991ccfaa0d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:52.841865 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-12538484-3b8a-45ca-bde2-93991ccfaa0d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:52.852992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-12538484-3b8a-45ca-bde2-93991ccfaa0d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:52.859050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:10:53.213086 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-60c4d8d4-0fb6-4726-bde1-eab3228327c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:53.217523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=99 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:10:53.257740 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-60c4d8d4-0fb6-4726-bde1-eab3228327c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1363 microversion: 2.1 time: 1.871298 Aug 07 01:10:53.285337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 42/78] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:51 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1363 bytes in 1886 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:53.323629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-229ca64e-9fe1-40df-8e58-d5a822b20d00 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:53.325686 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-229ca64e-9fe1-40df-8e58-d5a822b20d00 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:53.364805 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=86,nova_api:INSERT=7 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:10:53.384141 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-229ca64e-9fe1-40df-8e58-d5a822b20d00 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:53.389129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-229ca64e-9fe1-40df-8e58-d5a822b20d00 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:53.391761 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-229ca64e-9fe1-40df-8e58-d5a822b20d00 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:53.394139 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-229ca64e-9fe1-40df-8e58-d5a822b20d00 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:53.396015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-229ca64e-9fe1-40df-8e58-d5a822b20d00 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:53.399474 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-229ca64e-9fe1-40df-8e58-d5a822b20d00 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:54.070470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-12538484-3b8a-45ca-bde2-93991ccfaa0d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:54.087657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-12538484-3b8a-45ca-bde2-93991ccfaa0d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:54.092770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-12538484-3b8a-45ca-bde2-93991ccfaa0d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:54.098324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-12538484-3b8a-45ca-bde2-93991ccfaa0d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1405 microversion: 2.20 time: 1.292687 Aug 07 01:10:54.109263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 37/79] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:10:52 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1405 bytes in 1304 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:10:54.115158 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bbcaace0-99e3-47ae-a6f2-3bfe740288af tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:54.116130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bbcaace0-99e3-47ae-a6f2-3bfe740288af tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:54.130277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bbcaace0-99e3-47ae-a6f2-3bfe740288af tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:54.131130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bbcaace0-99e3-47ae-a6f2-3bfe740288af tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:54.131842 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bbcaace0-99e3-47ae-a6f2-3bfe740288af tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:54.133044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bbcaace0-99e3-47ae-a6f2-3bfe740288af tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:54.135055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bbcaace0-99e3-47ae-a6f2-3bfe740288af tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:54.136662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bbcaace0-99e3-47ae-a6f2-3bfe740288af tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:54.295868 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-229ca64e-9fe1-40df-8e58-d5a822b20d00 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:54.319442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-229ca64e-9fe1-40df-8e58-d5a822b20d00 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1367 microversion: 2.1 time: 1.032445 Aug 07 01:10:54.320138 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 43/80] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:53 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1367 bytes in 1034 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:54.327409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e439ba15-3ca3-4c44-9cbd-96d21f60611d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:54.329433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e439ba15-3ca3-4c44-9cbd-96d21f60611d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:54.340435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e439ba15-3ca3-4c44-9cbd-96d21f60611d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:54.340792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e439ba15-3ca3-4c44-9cbd-96d21f60611d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:54.341470 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e439ba15-3ca3-4c44-9cbd-96d21f60611d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:54.342353 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e439ba15-3ca3-4c44-9cbd-96d21f60611d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:54.343190 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e439ba15-3ca3-4c44-9cbd-96d21f60611d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:54.343775 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e439ba15-3ca3-4c44-9cbd-96d21f60611d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:54.492563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-bbcaace0-99e3-47ae-a6f2-3bfe740288af tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:54.523935 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bbcaace0-99e3-47ae-a6f2-3bfe740288af tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1314 microversion: 2.1 time: 0.410137 Aug 07 01:10:54.527401 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 38/81] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:54 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1314 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:54.637584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-4efeebaa-6376-4dfa-a251-1dfab851278e req-fa552a39-3f42-45fa-8b31-4bbadefe4632 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4f3f0774-3a34-4f19-9233-c4e55c7f5903", "tag": "9e5aa5e7-49cb-4c56-b9f5-3adf7c37f16a"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:10:54.675985 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-4efeebaa-6376-4dfa-a251-1dfab851278e req-fa552a39-3f42-45fa-8b31-4bbadefe4632 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:54.677148 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-4efeebaa-6376-4dfa-a251-1dfab851278e req-fa552a39-3f42-45fa-8b31-4bbadefe4632 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:54.677761 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-4efeebaa-6376-4dfa-a251-1dfab851278e req-fa552a39-3f42-45fa-8b31-4bbadefe4632 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:54.731187 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-4efeebaa-6376-4dfa-a251-1dfab851278e req-fa552a39-3f42-45fa-8b31-4bbadefe4632 service nova] Creating event network-changed:9e5aa5e7-49cb-4c56-b9f5-3adf7c37f16a for instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 on hv-cinder-maste Aug 07 01:10:54.749183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-4efeebaa-6376-4dfa-a251-1dfab851278e req-fa552a39-3f42-45fa-8b31-4bbadefe4632 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.118221 Aug 07 01:10:54.754667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 39/82] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:10:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:10:55.042319 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e439ba15-3ca3-4c44-9cbd-96d21f60611d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:55.057985 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e439ba15-3ca3-4c44-9cbd-96d21f60611d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1363 microversion: 2.1 time: 0.736409 Aug 07 01:10:55.063503 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 44/83] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:54 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1363 bytes in 743 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:55.142371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5fbd228d-e64d-40ce-8a47-39a940ad3215 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:55.145852 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5fbd228d-e64d-40ce-8a47-39a940ad3215 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:55.158323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fbd228d-e64d-40ce-8a47-39a940ad3215 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:55.159155 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fbd228d-e64d-40ce-8a47-39a940ad3215 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:55.161133 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fbd228d-e64d-40ce-8a47-39a940ad3215 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:55.163612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fbd228d-e64d-40ce-8a47-39a940ad3215 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:55.166542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fbd228d-e64d-40ce-8a47-39a940ad3215 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:55.167305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fbd228d-e64d-40ce-8a47-39a940ad3215 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:55.372674 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1330a0fd-d9a7-4cb1-8162-d2b96cf19982 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:55.374381 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1330a0fd-d9a7-4cb1-8162-d2b96cf19982 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:55.387918 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1330a0fd-d9a7-4cb1-8162-d2b96cf19982 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:55.388861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1330a0fd-d9a7-4cb1-8162-d2b96cf19982 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:55.390049 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1330a0fd-d9a7-4cb1-8162-d2b96cf19982 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:55.392642 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1330a0fd-d9a7-4cb1-8162-d2b96cf19982 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:55.393933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1330a0fd-d9a7-4cb1-8162-d2b96cf19982 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:55.394923 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1330a0fd-d9a7-4cb1-8162-d2b96cf19982 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:55.716268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1330a0fd-d9a7-4cb1-8162-d2b96cf19982 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:55.732874 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1330a0fd-d9a7-4cb1-8162-d2b96cf19982 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1367 microversion: 2.1 time: 0.364743 Aug 07 01:10:55.733933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 45/84] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:55 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1367 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:10:55.741165 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-38f135a7-1407-4d14-a72b-c570ed1774c1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:55.742188 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-38f135a7-1407-4d14-a72b-c570ed1774c1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:55.750990 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-38f135a7-1407-4d14-a72b-c570ed1774c1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:55.752002 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-38f135a7-1407-4d14-a72b-c570ed1774c1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:55.752827 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-38f135a7-1407-4d14-a72b-c570ed1774c1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:55.754318 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-38f135a7-1407-4d14-a72b-c570ed1774c1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:55.755314 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-38f135a7-1407-4d14-a72b-c570ed1774c1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:55.756022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-38f135a7-1407-4d14-a72b-c570ed1774c1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:55.864749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5fbd228d-e64d-40ce-8a47-39a940ad3215 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:55.883297 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5fbd228d-e64d-40ce-8a47-39a940ad3215 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:55.890078 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5fbd228d-e64d-40ce-8a47-39a940ad3215 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:55.892441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5fbd228d-e64d-40ce-8a47-39a940ad3215 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1405 microversion: 2.20 time: 0.757500 Aug 07 01:10:55.893544 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 40/85] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:10:55 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1405 bytes in 761 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:10:56.001837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-38f135a7-1407-4d14-a72b-c570ed1774c1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:56.018732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-38f135a7-1407-4d14-a72b-c570ed1774c1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1314 microversion: 2.1 time: 0.281820 Aug 07 01:10:56.021156 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 46/86] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:55 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1314 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:56.087725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1e44a301-2e1d-4e43-a2b1-24ca31e6abca tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:56.088758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1e44a301-2e1d-4e43-a2b1-24ca31e6abca tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:56.097960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e44a301-2e1d-4e43-a2b1-24ca31e6abca tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:56.098480 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e44a301-2e1d-4e43-a2b1-24ca31e6abca tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:56.098978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e44a301-2e1d-4e43-a2b1-24ca31e6abca tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:56.099764 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e44a301-2e1d-4e43-a2b1-24ca31e6abca tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:56.100295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e44a301-2e1d-4e43-a2b1-24ca31e6abca tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:56.101024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e44a301-2e1d-4e43-a2b1-24ca31e6abca tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:56.655098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1e44a301-2e1d-4e43-a2b1-24ca31e6abca tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:56.702960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1e44a301-2e1d-4e43-a2b1-24ca31e6abca tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1363 microversion: 2.1 time: 0.619102 Aug 07 01:10:56.704967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 41/87] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:56 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1363 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:56.769859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-735c3a92-9906-4738-b01d-7353ab02d96d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:56.770937 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-735c3a92-9906-4738-b01d-7353ab02d96d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:56.780365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-735c3a92-9906-4738-b01d-7353ab02d96d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:56.781063 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-735c3a92-9906-4738-b01d-7353ab02d96d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:56.781639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-735c3a92-9906-4738-b01d-7353ab02d96d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:56.783045 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-735c3a92-9906-4738-b01d-7353ab02d96d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:56.783790 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-735c3a92-9906-4738-b01d-7353ab02d96d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:56.784567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-735c3a92-9906-4738-b01d-7353ab02d96d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:56.874887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:10:56.930993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-eb2cd3e4-5f32-46d7-b1ca-982379fa9ba1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:56.932585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-eb2cd3e4-5f32-46d7-b1ca-982379fa9ba1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:56.974881 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb2cd3e4-5f32-46d7-b1ca-982379fa9ba1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:56.975787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb2cd3e4-5f32-46d7-b1ca-982379fa9ba1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:56.976674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb2cd3e4-5f32-46d7-b1ca-982379fa9ba1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:56.978449 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb2cd3e4-5f32-46d7-b1ca-982379fa9ba1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:56.979777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb2cd3e4-5f32-46d7-b1ca-982379fa9ba1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:56.981046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb2cd3e4-5f32-46d7-b1ca-982379fa9ba1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:57.549240 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-eb2cd3e4-5f32-46d7-b1ca-982379fa9ba1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:57.570517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-eb2cd3e4-5f32-46d7-b1ca-982379fa9ba1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:57.582399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-eb2cd3e4-5f32-46d7-b1ca-982379fa9ba1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:57.585957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-eb2cd3e4-5f32-46d7-b1ca-982379fa9ba1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1405 microversion: 2.20 time: 0.661611 Aug 07 01:10:57.588586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 42/88] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:10:56 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1405 bytes in 666 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:10:57.602100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a3f64c66-a87c-4fca-ac6c-e83100f2ce09 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:57.605290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a3f64c66-a87c-4fca-ac6c-e83100f2ce09 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:57.625245 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a3f64c66-a87c-4fca-ac6c-e83100f2ce09 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:57.626465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a3f64c66-a87c-4fca-ac6c-e83100f2ce09 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:57.627662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a3f64c66-a87c-4fca-ac6c-e83100f2ce09 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:57.630573 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a3f64c66-a87c-4fca-ac6c-e83100f2ce09 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:57.639358 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a3f64c66-a87c-4fca-ac6c-e83100f2ce09 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:57.640630 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a3f64c66-a87c-4fca-ac6c-e83100f2ce09 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:57.871945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:10:58.017121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-735c3a92-9906-4738-b01d-7353ab02d96d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:58.044406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-735c3a92-9906-4738-b01d-7353ab02d96d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1367 microversion: 2.1 time: 1.278999 Aug 07 01:10:58.044406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 47/89] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:56 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1367 bytes in 1281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:10:58.071286 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9311bfcb-9478-4248-84bf-13688670990a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:58.076477 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9311bfcb-9478-4248-84bf-13688670990a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:58.105506 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9311bfcb-9478-4248-84bf-13688670990a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:58.109241 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9311bfcb-9478-4248-84bf-13688670990a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:58.124158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9311bfcb-9478-4248-84bf-13688670990a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:58.126656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9311bfcb-9478-4248-84bf-13688670990a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:58.130562 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9311bfcb-9478-4248-84bf-13688670990a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:58.131242 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9311bfcb-9478-4248-84bf-13688670990a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:58.782103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9311bfcb-9478-4248-84bf-13688670990a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:58.799102 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a3f64c66-a87c-4fca-ac6c-e83100f2ce09 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:58.805707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9311bfcb-9478-4248-84bf-13688670990a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1363 microversion: 2.1 time: 0.760228 Aug 07 01:10:58.808600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 48/90] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:58 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1363 bytes in 764 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:58.828809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d46845f8-e6a2-46b5-a857-cd455e2f17db tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:58.830385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d46845f8-e6a2-46b5-a857-cd455e2f17db tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:58.832059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a3f64c66-a87c-4fca-ac6c-e83100f2ce09 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1314 microversion: 2.1 time: 1.239945 Aug 07 01:10:58.833201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 43/91] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:57 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1314 bytes in 1244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:10:58.846449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d46845f8-e6a2-46b5-a857-cd455e2f17db tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:58.872612 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d46845f8-e6a2-46b5-a857-cd455e2f17db tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:58.873613 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d46845f8-e6a2-46b5-a857-cd455e2f17db tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.024s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:58.880753 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d46845f8-e6a2-46b5-a857-cd455e2f17db tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:58.882042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d46845f8-e6a2-46b5-a857-cd455e2f17db tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:58.883243 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d46845f8-e6a2-46b5-a857-cd455e2f17db tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:59.091774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-edd81499-7c78-428b-a6df-b20400b0589f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:59.097860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-edd81499-7c78-428b-a6df-b20400b0589f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:59.120644 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-edd81499-7c78-428b-a6df-b20400b0589f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:59.121526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-edd81499-7c78-428b-a6df-b20400b0589f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:59.122560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-edd81499-7c78-428b-a6df-b20400b0589f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:59.123816 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-edd81499-7c78-428b-a6df-b20400b0589f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:59.137486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-edd81499-7c78-428b-a6df-b20400b0589f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:59.138296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-edd81499-7c78-428b-a6df-b20400b0589f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:59.747525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d46845f8-e6a2-46b5-a857-cd455e2f17db tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:59.760124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-edd81499-7c78-428b-a6df-b20400b0589f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:59.792945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-edd81499-7c78-428b-a6df-b20400b0589f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1367 microversion: 2.1 time: 0.707806 Aug 07 01:10:59.796363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d46845f8-e6a2-46b5-a857-cd455e2f17db tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:59.808043 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 44/92] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:59 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1367 bytes in 722 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:10:59.814726 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d46845f8-e6a2-46b5-a857-cd455e2f17db tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:10:59.817246 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d46845f8-e6a2-46b5-a857-cd455e2f17db tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1405 microversion: 2.20 time: 1.003779 Aug 07 01:10:59.825095 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 49/93] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:10:58 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1405 bytes in 1013 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:10:59.866878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a0ad78c2-a895-4408-aed0-8b14850c5492 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:59.870579 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a0ad78c2-a895-4408-aed0-8b14850c5492 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:59.890310 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-59e8ca8e-df57-4916-bea1-57612f98093a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:10:59.891283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-59e8ca8e-df57-4916-bea1-57612f98093a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:10:59.900755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0ad78c2-a895-4408-aed0-8b14850c5492 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:59.902470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0ad78c2-a895-4408-aed0-8b14850c5492 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:59.903351 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0ad78c2-a895-4408-aed0-8b14850c5492 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:59.908982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0ad78c2-a895-4408-aed0-8b14850c5492 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:59.909931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0ad78c2-a895-4408-aed0-8b14850c5492 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:59.924056 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59e8ca8e-df57-4916-bea1-57612f98093a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:59.924969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59e8ca8e-df57-4916-bea1-57612f98093a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:59.926052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59e8ca8e-df57-4916-bea1-57612f98093a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:59.927852 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59e8ca8e-df57-4916-bea1-57612f98093a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:10:59.932392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59e8ca8e-df57-4916-bea1-57612f98093a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:10:59.933180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59e8ca8e-df57-4916-bea1-57612f98093a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:10:59.933329 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0ad78c2-a895-4408-aed0-8b14850c5492 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.023s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:00.362207 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-59e8ca8e-df57-4916-bea1-57612f98093a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:00.380466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-59e8ca8e-df57-4916-bea1-57612f98093a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1314 microversion: 2.1 time: 0.494095 Aug 07 01:11:00.382693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 50/94] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:59 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1314 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:00.692284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a0ad78c2-a895-4408-aed0-8b14850c5492 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:00.714355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a0ad78c2-a895-4408-aed0-8b14850c5492 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1363 microversion: 2.1 time: 0.855154 Aug 07 01:11:00.717384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 45/95] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:10:59 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1363 bytes in 861 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:00.839585 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0542a3ca-0d7f-4a94-9f02-41b24e350bbf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:00.842870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0542a3ca-0d7f-4a94-9f02-41b24e350bbf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:00.857514 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0542a3ca-0d7f-4a94-9f02-41b24e350bbf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:00.858687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0542a3ca-0d7f-4a94-9f02-41b24e350bbf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:00.859593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0542a3ca-0d7f-4a94-9f02-41b24e350bbf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:00.875877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d73ce280-11da-4223-ac93-607a205a7119 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:00.877528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d73ce280-11da-4223-ac93-607a205a7119 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:00.878490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0542a3ca-0d7f-4a94-9f02-41b24e350bbf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:00.879641 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0542a3ca-0d7f-4a94-9f02-41b24e350bbf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:00.885833 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0542a3ca-0d7f-4a94-9f02-41b24e350bbf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:00.937236 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d73ce280-11da-4223-ac93-607a205a7119 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:00.938149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d73ce280-11da-4223-ac93-607a205a7119 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.025s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:00.938909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d73ce280-11da-4223-ac93-607a205a7119 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:00.940265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d73ce280-11da-4223-ac93-607a205a7119 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:00.941171 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d73ce280-11da-4223-ac93-607a205a7119 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:00.942261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d73ce280-11da-4223-ac93-607a205a7119 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:01.285303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d73ce280-11da-4223-ac93-607a205a7119 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:01.291244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0542a3ca-0d7f-4a94-9f02-41b24e350bbf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:01.319532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0542a3ca-0d7f-4a94-9f02-41b24e350bbf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1367 microversion: 2.1 time: 0.485670 Aug 07 01:11:01.321630 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d73ce280-11da-4223-ac93-607a205a7119 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:01.325172 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 51/96] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:00 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1367 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:01.325560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d73ce280-11da-4223-ac93-607a205a7119 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:01.327862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d73ce280-11da-4223-ac93-607a205a7119 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1405 microversion: 2.20 time: 0.460955 Aug 07 01:11:01.340838 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 46/97] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:11:00 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1405 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:11:01.414808 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-17fffbb2-bc29-46bd-bf66-af5008c6dcb7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:01.415976 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-17fffbb2-bc29-46bd-bf66-af5008c6dcb7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:01.428017 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17fffbb2-bc29-46bd-bf66-af5008c6dcb7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:01.428914 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17fffbb2-bc29-46bd-bf66-af5008c6dcb7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:01.429468 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17fffbb2-bc29-46bd-bf66-af5008c6dcb7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:01.430615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17fffbb2-bc29-46bd-bf66-af5008c6dcb7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:01.431716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17fffbb2-bc29-46bd-bf66-af5008c6dcb7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:01.433048 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17fffbb2-bc29-46bd-bf66-af5008c6dcb7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:01.769499 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-17fffbb2-bc29-46bd-bf66-af5008c6dcb7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:01.790344 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-17fffbb2-bc29-46bd-bf66-af5008c6dcb7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1314 microversion: 2.1 time: 0.382964 Aug 07 01:11:01.791918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 47/98] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:01 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1314 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:01.805780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-46709d14-3f55-4a3b-96a1-ab6107555f12 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:01.806521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-46709d14-3f55-4a3b-96a1-ab6107555f12 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:01.818747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-46709d14-3f55-4a3b-96a1-ab6107555f12 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:01.819312 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-46709d14-3f55-4a3b-96a1-ab6107555f12 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:01.820015 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-46709d14-3f55-4a3b-96a1-ab6107555f12 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:01.821975 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-46709d14-3f55-4a3b-96a1-ab6107555f12 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:01.822711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-46709d14-3f55-4a3b-96a1-ab6107555f12 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:01.823149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-46709d14-3f55-4a3b-96a1-ab6107555f12 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:02.020907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-46709d14-3f55-4a3b-96a1-ab6107555f12 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:02.035522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-46709d14-3f55-4a3b-96a1-ab6107555f12 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1363 microversion: 2.1 time: 0.235483 Aug 07 01:11:02.039043 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 48/99] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:01 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1363 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:02.125194 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-2bc4cd01-0f24-4d87-aff8-37c75263e22e req-ff5f26a0-4b87-4bd4-acce-78592d426eea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "93936488-825c-4db2-b648-ac856631c434", "name": "network-vif-plugged", "status": "completed", "tag": "a2435539-4100-4730-a912-3afae7d39093"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:02.141249 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-2bc4cd01-0f24-4d87-aff8-37c75263e22e req-ff5f26a0-4b87-4bd4-acce-78592d426eea service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:02.141896 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-2bc4cd01-0f24-4d87-aff8-37c75263e22e req-ff5f26a0-4b87-4bd4-acce-78592d426eea service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:02.142334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-2bc4cd01-0f24-4d87-aff8-37c75263e22e req-ff5f26a0-4b87-4bd4-acce-78592d426eea service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:02.159595 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-2bc4cd01-0f24-4d87-aff8-37c75263e22e req-ff5f26a0-4b87-4bd4-acce-78592d426eea service nova] Creating event network-vif-plugged:a2435539-4100-4730-a912-3afae7d39093 for instance 93936488-825c-4db2-b648-ac856631c434 on hv-cinder-maste Aug 07 01:11:02.167271 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-2bc4cd01-0f24-4d87-aff8-37c75263e22e req-ff5f26a0-4b87-4bd4-acce-78592d426eea service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.839513 Aug 07 01:11:02.169999 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 52/100] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:11:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 844 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:11:02.357617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ef9e0cde-22ce-451a-ac5a-f7b17e886106 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:02.358634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ef9e0cde-22ce-451a-ac5a-f7b17e886106 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:02.376762 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-67c0f674-9ea7-4bda-8bf6-0015f83d5e49 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:02.382965 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-67c0f674-9ea7-4bda-8bf6-0015f83d5e49 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:02.383980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef9e0cde-22ce-451a-ac5a-f7b17e886106 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:02.388141 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef9e0cde-22ce-451a-ac5a-f7b17e886106 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:02.389430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef9e0cde-22ce-451a-ac5a-f7b17e886106 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:02.392952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef9e0cde-22ce-451a-ac5a-f7b17e886106 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:02.395404 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef9e0cde-22ce-451a-ac5a-f7b17e886106 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:02.399757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef9e0cde-22ce-451a-ac5a-f7b17e886106 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:02.411390 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67c0f674-9ea7-4bda-8bf6-0015f83d5e49 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:02.413196 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67c0f674-9ea7-4bda-8bf6-0015f83d5e49 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:02.414432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67c0f674-9ea7-4bda-8bf6-0015f83d5e49 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:02.421374 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67c0f674-9ea7-4bda-8bf6-0015f83d5e49 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:02.423261 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67c0f674-9ea7-4bda-8bf6-0015f83d5e49 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:02.425193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67c0f674-9ea7-4bda-8bf6-0015f83d5e49 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:02.827585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ef9e0cde-22ce-451a-ac5a-f7b17e886106 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:02.843305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ef9e0cde-22ce-451a-ac5a-f7b17e886106 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1367 microversion: 2.1 time: 0.495088 Aug 07 01:11:02.846206 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 49/101] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:02 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1367 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:02.858183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1cbd4b7a-9ab5-4f55-af64-2cd8c86bb9d1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:02.860594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1cbd4b7a-9ab5-4f55-af64-2cd8c86bb9d1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:02.894678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1cbd4b7a-9ab5-4f55-af64-2cd8c86bb9d1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:02.895895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1cbd4b7a-9ab5-4f55-af64-2cd8c86bb9d1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:02.896562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1cbd4b7a-9ab5-4f55-af64-2cd8c86bb9d1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:02.897921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1cbd4b7a-9ab5-4f55-af64-2cd8c86bb9d1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:02.898312 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1cbd4b7a-9ab5-4f55-af64-2cd8c86bb9d1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:02.898815 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1cbd4b7a-9ab5-4f55-af64-2cd8c86bb9d1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:03.172352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-67c0f674-9ea7-4bda-8bf6-0015f83d5e49 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:03.188443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-67c0f674-9ea7-4bda-8bf6-0015f83d5e49 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:03.194241 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-67c0f674-9ea7-4bda-8bf6-0015f83d5e49 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:03.211968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-67c0f674-9ea7-4bda-8bf6-0015f83d5e49 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1405 microversion: 2.20 time: 0.831004 Aug 07 01:11:03.214623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 53/102] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:11:02 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1405 bytes in 852 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:11:03.242070 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3c9dd738-f7d0-480b-a826-29e0edeca906 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:03.242691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3c9dd738-f7d0-480b-a826-29e0edeca906 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:03.265234 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3c9dd738-f7d0-480b-a826-29e0edeca906 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:03.265953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3c9dd738-f7d0-480b-a826-29e0edeca906 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:03.266552 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3c9dd738-f7d0-480b-a826-29e0edeca906 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:03.267325 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3c9dd738-f7d0-480b-a826-29e0edeca906 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:03.267811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3c9dd738-f7d0-480b-a826-29e0edeca906 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:03.268268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3c9dd738-f7d0-480b-a826-29e0edeca906 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:03.523377 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1cbd4b7a-9ab5-4f55-af64-2cd8c86bb9d1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:03.550587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1cbd4b7a-9ab5-4f55-af64-2cd8c86bb9d1 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1314 microversion: 2.1 time: 0.700870 Aug 07 01:11:03.559002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 50/103] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:02 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1314 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:03.805560 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3c9dd738-f7d0-480b-a826-29e0edeca906 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:03.837314 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3c9dd738-f7d0-480b-a826-29e0edeca906 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1363 microversion: 2.1 time: 0.604409 Aug 07 01:11:03.839442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 54/104] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:03 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1363 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:03.890230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-63f98b00-6a7a-49de-a760-b3db6f6684d1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:03.891294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-63f98b00-6a7a-49de-a760-b3db6f6684d1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:03.902839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63f98b00-6a7a-49de-a760-b3db6f6684d1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:03.904647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63f98b00-6a7a-49de-a760-b3db6f6684d1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:03.907302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63f98b00-6a7a-49de-a760-b3db6f6684d1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:03.910208 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63f98b00-6a7a-49de-a760-b3db6f6684d1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:03.911371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63f98b00-6a7a-49de-a760-b3db6f6684d1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:03.913086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63f98b00-6a7a-49de-a760-b3db6f6684d1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:04.233193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-63f98b00-6a7a-49de-a760-b3db6f6684d1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:04.252723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-91edc2e6-df72-447a-a7cd-f488f0c57bf2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:04.259003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-91edc2e6-df72-447a-a7cd-f488f0c57bf2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:04.287143 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-63f98b00-6a7a-49de-a760-b3db6f6684d1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1367 microversion: 2.1 time: 0.399943 Aug 07 01:11:04.289248 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 51/105] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:03 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1367 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:04.292528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-91edc2e6-df72-447a-a7cd-f488f0c57bf2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:04.294047 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-91edc2e6-df72-447a-a7cd-f488f0c57bf2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:04.294262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-91edc2e6-df72-447a-a7cd-f488f0c57bf2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:04.297587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-91edc2e6-df72-447a-a7cd-f488f0c57bf2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:04.298493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-91edc2e6-df72-447a-a7cd-f488f0c57bf2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:04.302915 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-91edc2e6-df72-447a-a7cd-f488f0c57bf2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:04.314978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-ef252428-4972-4c77-a060-3ff38a926bdf req-4c88dfa5-de99-4451-ad02-b1a53704ed29 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7f60fb22-d17a-4b7c-9b7d-95543be1907e", "name": "network-vif-plugged", "status": "completed", "tag": "9ab59ce3-ed0d-4a8e-b76b-adeff559725a"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:04.363886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-ef252428-4972-4c77-a060-3ff38a926bdf req-4c88dfa5-de99-4451-ad02-b1a53704ed29 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:04.364773 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-ef252428-4972-4c77-a060-3ff38a926bdf req-4c88dfa5-de99-4451-ad02-b1a53704ed29 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:04.366200 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-ef252428-4972-4c77-a060-3ff38a926bdf req-4c88dfa5-de99-4451-ad02-b1a53704ed29 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:04.419128 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-ef252428-4972-4c77-a060-3ff38a926bdf req-4c88dfa5-de99-4451-ad02-b1a53704ed29 service nova] Creating event network-vif-plugged:9ab59ce3-ed0d-4a8e-b76b-adeff559725a for instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e on hv-cinder-maste Aug 07 01:11:04.445008 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-ef252428-4972-4c77-a060-3ff38a926bdf req-4c88dfa5-de99-4451-ad02-b1a53704ed29 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.152398 Aug 07 01:11:04.446808 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 52/106] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:11:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:11:04.616156 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a805d731-3498-4d05-ac1f-65973df8e7f7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:04.616920 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a805d731-3498-4d05-ac1f-65973df8e7f7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:04.640288 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a805d731-3498-4d05-ac1f-65973df8e7f7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:04.641150 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a805d731-3498-4d05-ac1f-65973df8e7f7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:04.646271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a805d731-3498-4d05-ac1f-65973df8e7f7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:04.647569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a805d731-3498-4d05-ac1f-65973df8e7f7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:04.649519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a805d731-3498-4d05-ac1f-65973df8e7f7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:04.653358 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a805d731-3498-4d05-ac1f-65973df8e7f7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:04.764771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-91edc2e6-df72-447a-a7cd-f488f0c57bf2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:04.788589 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-91edc2e6-df72-447a-a7cd-f488f0c57bf2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:04.792909 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-91edc2e6-df72-447a-a7cd-f488f0c57bf2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:04.795468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-91edc2e6-df72-447a-a7cd-f488f0c57bf2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1405 microversion: 2.20 time: 0.558740 Aug 07 01:11:04.803392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 55/107] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:11:04 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1405 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:11:04.890149 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9f21ee60-3c4d-4b7b-9f96-34f702b0b604 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:04.892738 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9f21ee60-3c4d-4b7b-9f96-34f702b0b604 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:04.922235 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9f21ee60-3c4d-4b7b-9f96-34f702b0b604 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:04.929165 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9f21ee60-3c4d-4b7b-9f96-34f702b0b604 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:04.930044 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9f21ee60-3c4d-4b7b-9f96-34f702b0b604 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:04.931390 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9f21ee60-3c4d-4b7b-9f96-34f702b0b604 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:04.932219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9f21ee60-3c4d-4b7b-9f96-34f702b0b604 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:04.933048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9f21ee60-3c4d-4b7b-9f96-34f702b0b604 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:04.947095 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a805d731-3498-4d05-ac1f-65973df8e7f7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:04.960727 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a805d731-3498-4d05-ac1f-65973df8e7f7 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1314 microversion: 2.1 time: 0.348004 Aug 07 01:11:04.962022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 53/108] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:04 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1314 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:05.291204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9f21ee60-3c4d-4b7b-9f96-34f702b0b604 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:05.312756 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9f21ee60-3c4d-4b7b-9f96-34f702b0b604 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1363 microversion: 2.1 time: 0.438985 Aug 07 01:11:05.314715 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 56/109] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:04 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1363 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:05.335670 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-715e2458-1841-49ca-a282-748f4c34d2ad tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:05.337358 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-715e2458-1841-49ca-a282-748f4c34d2ad tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:05.351594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-715e2458-1841-49ca-a282-748f4c34d2ad tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:05.352483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-715e2458-1841-49ca-a282-748f4c34d2ad tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:05.353386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-715e2458-1841-49ca-a282-748f4c34d2ad tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:05.354646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-715e2458-1841-49ca-a282-748f4c34d2ad tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:05.355649 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-715e2458-1841-49ca-a282-748f4c34d2ad tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:05.356442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-715e2458-1841-49ca-a282-748f4c34d2ad tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:05.592270 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-715e2458-1841-49ca-a282-748f4c34d2ad tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:05.624642 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-715e2458-1841-49ca-a282-748f4c34d2ad tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1367 microversion: 2.1 time: 0.300347 Aug 07 01:11:05.625818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 54/110] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:05 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1367 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:05.854762 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-76ef250e-2f5d-4d71-8740-97da6c0e046c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:05.855798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-76ef250e-2f5d-4d71-8740-97da6c0e046c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:05.871570 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-76ef250e-2f5d-4d71-8740-97da6c0e046c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:05.872421 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-76ef250e-2f5d-4d71-8740-97da6c0e046c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:05.873396 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-76ef250e-2f5d-4d71-8740-97da6c0e046c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:05.875050 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-76ef250e-2f5d-4d71-8740-97da6c0e046c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:05.876137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-76ef250e-2f5d-4d71-8740-97da6c0e046c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:05.877545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-76ef250e-2f5d-4d71-8740-97da6c0e046c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:06.005653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9bf0262a-2ba4-4342-aec0-24d94b35d30c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:06.006565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9bf0262a-2ba4-4342-aec0-24d94b35d30c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:06.051418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9bf0262a-2ba4-4342-aec0-24d94b35d30c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:06.052904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9bf0262a-2ba4-4342-aec0-24d94b35d30c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:06.053987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9bf0262a-2ba4-4342-aec0-24d94b35d30c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:06.058637 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9bf0262a-2ba4-4342-aec0-24d94b35d30c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:06.062508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9bf0262a-2ba4-4342-aec0-24d94b35d30c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:06.066688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9bf0262a-2ba4-4342-aec0-24d94b35d30c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:06.078041 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:06.222020 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-76ef250e-2f5d-4d71-8740-97da6c0e046c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:06.245489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-76ef250e-2f5d-4d71-8740-97da6c0e046c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:06.249279 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-76ef250e-2f5d-4d71-8740-97da6c0e046c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:06.252295 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-76ef250e-2f5d-4d71-8740-97da6c0e046c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1405 microversion: 2.20 time: 0.408444 Aug 07 01:11:06.254784 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 57/111] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:11:05 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1405 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:11:06.375547 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3583e0ae-ae26-4fd9-baee-ea1843c6d715 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:06.376546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3583e0ae-ae26-4fd9-baee-ea1843c6d715 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:06.394885 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9bf0262a-2ba4-4342-aec0-24d94b35d30c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:06.401039 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3583e0ae-ae26-4fd9-baee-ea1843c6d715 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:06.401914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3583e0ae-ae26-4fd9-baee-ea1843c6d715 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:06.402671 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3583e0ae-ae26-4fd9-baee-ea1843c6d715 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:06.406437 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3583e0ae-ae26-4fd9-baee-ea1843c6d715 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:06.411766 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3583e0ae-ae26-4fd9-baee-ea1843c6d715 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:06.412541 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3583e0ae-ae26-4fd9-baee-ea1843c6d715 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:06.420117 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9bf0262a-2ba4-4342-aec0-24d94b35d30c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1501 microversion: 2.1 time: 0.424988 Aug 07 01:11:06.422783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 55/112] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:05 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1501 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:06.474855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-fe2caeb1-4a6f-4e94-a9cb-87a47da474b4 req-2b4ea812-49c1-46d7-ba52-603ce39431db service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "96f9a9ce-f29e-48be-a36b-8518126a09ee", "name": "network-vif-plugged", "status": "completed", "tag": "f71ffe87-943a-40bf-94f2-b17532b7092c"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:06.494418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-fe2caeb1-4a6f-4e94-a9cb-87a47da474b4 req-2b4ea812-49c1-46d7-ba52-603ce39431db service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:06.496775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-fe2caeb1-4a6f-4e94-a9cb-87a47da474b4 req-2b4ea812-49c1-46d7-ba52-603ce39431db service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:06.497833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-fe2caeb1-4a6f-4e94-a9cb-87a47da474b4 req-2b4ea812-49c1-46d7-ba52-603ce39431db service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:06.541077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-fe2caeb1-4a6f-4e94-a9cb-87a47da474b4 req-2b4ea812-49c1-46d7-ba52-603ce39431db service nova] Creating event network-vif-plugged:f71ffe87-943a-40bf-94f2-b17532b7092c for instance 96f9a9ce-f29e-48be-a36b-8518126a09ee on hv-cinder-maste Aug 07 01:11:06.564518 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-fe2caeb1-4a6f-4e94-a9cb-87a47da474b4 req-2b4ea812-49c1-46d7-ba52-603ce39431db service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.096381 Aug 07 01:11:06.574637 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 56/113] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:11:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:11:06.659078 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b24b2052-904e-4a07-bbe0-8bbd7347fa1e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:06.669557 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b24b2052-904e-4a07-bbe0-8bbd7347fa1e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:06.733129 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b24b2052-904e-4a07-bbe0-8bbd7347fa1e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:06.736969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b24b2052-904e-4a07-bbe0-8bbd7347fa1e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:06.738689 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b24b2052-904e-4a07-bbe0-8bbd7347fa1e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:06.739149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b24b2052-904e-4a07-bbe0-8bbd7347fa1e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:06.740011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b24b2052-904e-4a07-bbe0-8bbd7347fa1e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:06.740816 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b24b2052-904e-4a07-bbe0-8bbd7347fa1e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:07.068856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3583e0ae-ae26-4fd9-baee-ea1843c6d715 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:07.103454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3583e0ae-ae26-4fd9-baee-ea1843c6d715 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1363 microversion: 2.1 time: 0.758349 Aug 07 01:11:07.112872 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 58/114] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:06 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1363 bytes in 769 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:07.309863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-59c90653-5797-4925-b509-0dac78ed0a47 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:07.328666 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-59c90653-5797-4925-b509-0dac78ed0a47 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:07.371937 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59c90653-5797-4925-b509-0dac78ed0a47 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:07.377441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59c90653-5797-4925-b509-0dac78ed0a47 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:07.380064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59c90653-5797-4925-b509-0dac78ed0a47 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:07.381328 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59c90653-5797-4925-b509-0dac78ed0a47 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:07.382116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59c90653-5797-4925-b509-0dac78ed0a47 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:07.382888 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59c90653-5797-4925-b509-0dac78ed0a47 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:07.413763 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b24b2052-904e-4a07-bbe0-8bbd7347fa1e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:07.456588 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b24b2052-904e-4a07-bbe0-8bbd7347fa1e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1367 microversion: 2.1 time: 0.795614 Aug 07 01:11:07.464510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 57/115] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:06 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1367 bytes in 818 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:08.067626 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-59c90653-5797-4925-b509-0dac78ed0a47 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:08.099189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-59c90653-5797-4925-b509-0dac78ed0a47 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:08.104048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-59c90653-5797-4925-b509-0dac78ed0a47 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:08.113172 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-59c90653-5797-4925-b509-0dac78ed0a47 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1598 microversion: 2.20 time: 0.823743 Aug 07 01:11:08.115154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 59/116] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:11:07 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1598 bytes in 828 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:11:08.156085 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-71a5fe38-560a-4a31-8db4-7893f12ac447 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:08.185800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c6a8d8c5-31a9-4da1-9a57-5224f9057176 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:08.186810 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c6a8d8c5-31a9-4da1-9a57-5224f9057176 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:08.191158 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-71a5fe38-560a-4a31-8db4-7893f12ac447 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:08.204870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6a8d8c5-31a9-4da1-9a57-5224f9057176 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:08.205404 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6a8d8c5-31a9-4da1-9a57-5224f9057176 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:08.205804 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6a8d8c5-31a9-4da1-9a57-5224f9057176 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:08.207070 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6a8d8c5-31a9-4da1-9a57-5224f9057176 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:08.214443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6a8d8c5-31a9-4da1-9a57-5224f9057176 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:08.214443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6a8d8c5-31a9-4da1-9a57-5224f9057176 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:08.214741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71a5fe38-560a-4a31-8db4-7893f12ac447 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:08.214741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71a5fe38-560a-4a31-8db4-7893f12ac447 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:08.215096 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71a5fe38-560a-4a31-8db4-7893f12ac447 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:08.216867 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71a5fe38-560a-4a31-8db4-7893f12ac447 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:08.217952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71a5fe38-560a-4a31-8db4-7893f12ac447 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:08.219105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71a5fe38-560a-4a31-8db4-7893f12ac447 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:08.317822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:08.482305 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c6a8d8c5-31a9-4da1-9a57-5224f9057176 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.323721 Aug 07 01:11:08.483863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 60/117] 10.222.0.26 () {60 vars in 1316 bytes} [Sun Aug 7 01:11:08 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/os-interface => generated 285 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:11:08.516960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-71a5fe38-560a-4a31-8db4-7893f12ac447 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:08.534921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-71a5fe38-560a-4a31-8db4-7893f12ac447 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1363 microversion: 2.1 time: 0.393378 Aug 07 01:11:08.536783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 58/118] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:08 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1363 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:08.545632 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a86a6728-a3ff-42d2-ab1d-719da2b46c73 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:08.558499 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a86a6728-a3ff-42d2-ab1d-719da2b46c73 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:08.603603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-98de55bf-e59d-4894-89d8-a1f709b47ee2 req-e3a1af07-6539-4f9c-ad76-dd64754c083d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4f3f0774-3a34-4f19-9233-c4e55c7f5903", "name": "network-vif-plugged", "status": "completed", "tag": "9e5aa5e7-49cb-4c56-b9f5-3adf7c37f16a"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:08.604413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a86a6728-a3ff-42d2-ab1d-719da2b46c73 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:08.605339 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a86a6728-a3ff-42d2-ab1d-719da2b46c73 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:08.606128 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a86a6728-a3ff-42d2-ab1d-719da2b46c73 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:08.607400 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a86a6728-a3ff-42d2-ab1d-719da2b46c73 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:08.608226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a86a6728-a3ff-42d2-ab1d-719da2b46c73 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:08.609044 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a86a6728-a3ff-42d2-ab1d-719da2b46c73 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:08.667907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-98de55bf-e59d-4894-89d8-a1f709b47ee2 req-e3a1af07-6539-4f9c-ad76-dd64754c083d service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:08.674436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-98de55bf-e59d-4894-89d8-a1f709b47ee2 req-e3a1af07-6539-4f9c-ad76-dd64754c083d service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:08.677496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-98de55bf-e59d-4894-89d8-a1f709b47ee2 req-e3a1af07-6539-4f9c-ad76-dd64754c083d service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:08.716181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-98de55bf-e59d-4894-89d8-a1f709b47ee2 req-e3a1af07-6539-4f9c-ad76-dd64754c083d service nova] Creating event network-vif-plugged:9e5aa5e7-49cb-4c56-b9f5-3adf7c37f16a for instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 on hv-cinder-maste Aug 07 01:11:08.733154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-98de55bf-e59d-4894-89d8-a1f709b47ee2 req-e3a1af07-6539-4f9c-ad76-dd64754c083d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.133753 Aug 07 01:11:08.738598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 59/119] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:11:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:11:08.783269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1c3d75f3-167f-4fce-bba6-4eb9eb3d6717 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "90e739bd-4d8a-4b9f-a63d-49b02c87597c"}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:08.796432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1c3d75f3-167f-4fce-bba6-4eb9eb3d6717 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:08.818530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c3d75f3-167f-4fce-bba6-4eb9eb3d6717 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:08.819034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c3d75f3-167f-4fce-bba6-4eb9eb3d6717 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:08.824388 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c3d75f3-167f-4fce-bba6-4eb9eb3d6717 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:08.830879 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c3d75f3-167f-4fce-bba6-4eb9eb3d6717 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:08.830879 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c3d75f3-167f-4fce-bba6-4eb9eb3d6717 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:08.832050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c3d75f3-167f-4fce-bba6-4eb9eb3d6717 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:09.510890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a86a6728-a3ff-42d2-ab1d-719da2b46c73 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:09.526785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a86a6728-a3ff-42d2-ab1d-719da2b46c73 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1367 microversion: 2.1 time: 1.019351 Aug 07 01:11:09.527751 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 61/120] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:08 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1367 bytes in 1021 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:09.572490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ade490e2-536f-4824-afb3-7badff3c0d78 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:09.573516 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ade490e2-536f-4824-afb3-7badff3c0d78 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:09.586553 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ade490e2-536f-4824-afb3-7badff3c0d78 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:09.588780 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ade490e2-536f-4824-afb3-7badff3c0d78 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:09.591024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ade490e2-536f-4824-afb3-7badff3c0d78 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:09.592047 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ade490e2-536f-4824-afb3-7badff3c0d78 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:09.596769 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ade490e2-536f-4824-afb3-7badff3c0d78 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:09.598051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ade490e2-536f-4824-afb3-7badff3c0d78 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:10.170047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1c3d75f3-167f-4fce-bba6-4eb9eb3d6717 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "POST /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.429253 Aug 07 01:11:10.170882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 60/121] 10.222.0.26 () {60 vars in 1323 bytes} [Sun Aug 7 01:11:08 2022] POST /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434/os-volume_attachments => generated 194 bytes in 1432 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:11:10.578180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-867b9136-b05f-4d12-96a6-881773c77543 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:10.580373 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-867b9136-b05f-4d12-96a6-881773c77543 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:10.594079 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-867b9136-b05f-4d12-96a6-881773c77543 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:10.595017 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-867b9136-b05f-4d12-96a6-881773c77543 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:10.596391 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-867b9136-b05f-4d12-96a6-881773c77543 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:10.597488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-867b9136-b05f-4d12-96a6-881773c77543 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:10.598648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-867b9136-b05f-4d12-96a6-881773c77543 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:10.599330 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-867b9136-b05f-4d12-96a6-881773c77543 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:10.730080 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ade490e2-536f-4824-afb3-7badff3c0d78 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:10.749457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ade490e2-536f-4824-afb3-7badff3c0d78 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1363 microversion: 2.1 time: 1.181224 Aug 07 01:11:10.750548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 62/122] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:09 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1363 bytes in 1183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:11.405714 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-867b9136-b05f-4d12-96a6-881773c77543 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:11.427860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-867b9136-b05f-4d12-96a6-881773c77543 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1558 microversion: 2.1 time: 0.855384 Aug 07 01:11:11.433539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 61/123] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:10 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1558 bytes in 863 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:11.470045 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-83199253-2a2a-4405-ae86-e5c0c5501ee1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:11.472388 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-83199253-2a2a-4405-ae86-e5c0c5501ee1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:11.485332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-83199253-2a2a-4405-ae86-e5c0c5501ee1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:11.486437 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-83199253-2a2a-4405-ae86-e5c0c5501ee1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:11.487480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-83199253-2a2a-4405-ae86-e5c0c5501ee1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:11.489585 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-83199253-2a2a-4405-ae86-e5c0c5501ee1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:11.490723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-83199253-2a2a-4405-ae86-e5c0c5501ee1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:11.491842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-83199253-2a2a-4405-ae86-e5c0c5501ee1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:11.782749 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-83199253-2a2a-4405-ae86-e5c0c5501ee1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.320529 Aug 07 01:11:11.785889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 63/124] 10.222.0.26 () {58 vars in 1275 bytes} [Sun Aug 7 01:11:11 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee/os-interface => generated 285 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:11:11.788419 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2428d088-9d98-4535-a0d9-81c9c8442d6e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:11.789425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2428d088-9d98-4535-a0d9-81c9c8442d6e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:11.817430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2428d088-9d98-4535-a0d9-81c9c8442d6e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:11.819954 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2428d088-9d98-4535-a0d9-81c9c8442d6e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:11.824428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2428d088-9d98-4535-a0d9-81c9c8442d6e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:11.827712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2428d088-9d98-4535-a0d9-81c9c8442d6e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:11.830419 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2428d088-9d98-4535-a0d9-81c9c8442d6e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:11.836402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2428d088-9d98-4535-a0d9-81c9c8442d6e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:12.311923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2428d088-9d98-4535-a0d9-81c9c8442d6e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:12.355213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2428d088-9d98-4535-a0d9-81c9c8442d6e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1363 microversion: 2.1 time: 0.571951 Aug 07 01:11:12.356773 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 62/125] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:11 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1363 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:13.097825 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-c11c310c-014c-4669-b544-c67151a57eeb req-a2e55227-c68d-4b84-bd18-a1a4d118b89d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7f60fb22-d17a-4b7c-9b7d-95543be1907e", "tag": "9ab59ce3-ed0d-4a8e-b76b-adeff559725a"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:13.121837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c11c310c-014c-4669-b544-c67151a57eeb req-a2e55227-c68d-4b84-bd18-a1a4d118b89d service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:13.122688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c11c310c-014c-4669-b544-c67151a57eeb req-a2e55227-c68d-4b84-bd18-a1a4d118b89d service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:13.123542 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c11c310c-014c-4669-b544-c67151a57eeb req-a2e55227-c68d-4b84-bd18-a1a4d118b89d service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:13.145439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-c11c310c-014c-4669-b544-c67151a57eeb req-a2e55227-c68d-4b84-bd18-a1a4d118b89d service nova] Creating event network-changed:9ab59ce3-ed0d-4a8e-b76b-adeff559725a for instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e on hv-cinder-maste Aug 07 01:11:13.156748 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-c11c310c-014c-4669-b544-c67151a57eeb req-a2e55227-c68d-4b84-bd18-a1a4d118b89d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065996 Aug 07 01:11:13.158503 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 64/126] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:11:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:11:13.401601 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b59386b2-269e-4e0a-963b-8207bbb1e88e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:13.407483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b59386b2-269e-4e0a-963b-8207bbb1e88e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:13.432849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b59386b2-269e-4e0a-963b-8207bbb1e88e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:13.433681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b59386b2-269e-4e0a-963b-8207bbb1e88e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:13.434438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b59386b2-269e-4e0a-963b-8207bbb1e88e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:13.435569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b59386b2-269e-4e0a-963b-8207bbb1e88e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:13.436396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b59386b2-269e-4e0a-963b-8207bbb1e88e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:13.437125 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b59386b2-269e-4e0a-963b-8207bbb1e88e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:13.897882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b59386b2-269e-4e0a-963b-8207bbb1e88e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:13.918229 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b59386b2-269e-4e0a-963b-8207bbb1e88e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1549 microversion: 2.1 time: 0.524173 Aug 07 01:11:13.918870 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 63/127] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:13 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1549 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:13.948426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d1a39569-f404-408a-9c30-e61d5094ca37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:13.949602 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d1a39569-f404-408a-9c30-e61d5094ca37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:13.965978 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1a39569-f404-408a-9c30-e61d5094ca37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:13.967057 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1a39569-f404-408a-9c30-e61d5094ca37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:13.967832 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1a39569-f404-408a-9c30-e61d5094ca37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:13.969373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1a39569-f404-408a-9c30-e61d5094ca37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:13.971711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1a39569-f404-408a-9c30-e61d5094ca37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:13.974002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-2061901750", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "networks": [{"uuid": "c801bdeb-0c37-4d5c-9982-2308bc5204c7"}]}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:13.975539 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1a39569-f404-408a-9c30-e61d5094ca37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:14.132511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.network.neutron [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] validate_networks() for [('c801bdeb-0c37-4d5c-9982-2308bc5204c7', None, None, None, None, None)] {{(pid=117983) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:11:15.048923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Flavor limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:11:15.049718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Image limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:11:15.051665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Flavor pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:11:15.052721 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Image pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:11:15.053467 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:11:15.078771 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:15.079743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:15.080579 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:15.082038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:15.082952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:15.083640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:15.119150 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Getting quotas for project 1c3de7a94e61473aa759f03c19119775. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:11:15.128091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Getting quotas for user dfc9712739724c91bd9dd5fdaaebd642 and project 1c3de7a94e61473aa759f03c19119775. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:11:15.153655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Going to run 1 instances... {{(pid=117983) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:11:15.158369 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117983) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:11:15.193004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d1a39569-f404-408a-9c30-e61d5094ca37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/os-interface" status: 200 len: 284 microversion: 2.1 time: 1.240713 Aug 07 01:11:15.197058 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 65/128] 10.222.0.26 () {58 vars in 1275 bytes} [Sun Aug 7 01:11:13 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/os-interface => generated 284 bytes in 1254 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:11:15.298372 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6695ed6a-2add-4e0a-8ede-adcaa86c32c6 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.334796 Aug 07 01:11:15.311293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 64/129] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:11:13 2022] POST /compute/v2.1/servers => generated 380 bytes in 1350 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:11:15.319811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-64ca854a-8514-4779-b67d-cbfe5c6c1661 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:15.329616 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-64ca854a-8514-4779-b67d-cbfe5c6c1661 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:15.922298 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-64ca854a-8514-4779-b67d-cbfe5c6c1661 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:15.931711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-64ca854a-8514-4779-b67d-cbfe5c6c1661 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1214 microversion: 2.1 time: 0.619051 Aug 07 01:11:15.932387 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 66/130] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:15 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1214 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:16.958931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1b8bcd83-1ec7-42bc-88c9-0984e5759681 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:16.959684 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1b8bcd83-1ec7-42bc-88c9-0984e5759681 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:16.979335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=79,nova_api:INSERT=3 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:16.983326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b8bcd83-1ec7-42bc-88c9-0984e5759681 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:16.983910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b8bcd83-1ec7-42bc-88c9-0984e5759681 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:16.984574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b8bcd83-1ec7-42bc-88c9-0984e5759681 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:16.985445 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b8bcd83-1ec7-42bc-88c9-0984e5759681 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:16.986266 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b8bcd83-1ec7-42bc-88c9-0984e5759681 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:16.993675 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b8bcd83-1ec7-42bc-88c9-0984e5759681 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:17.218228 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1b8bcd83-1ec7-42bc-88c9-0984e5759681 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:17.228296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-1576ab45-502b-4599-a031-85727d2b18ac req-41692e95-27e8-4068-9e54-03e5dce635dd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "96f9a9ce-f29e-48be-a36b-8518126a09ee", "tag": "f71ffe87-943a-40bf-94f2-b17532b7092c"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:17.236758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1b8bcd83-1ec7-42bc-88c9-0984e5759681 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1266 microversion: 2.1 time: 0.283043 Aug 07 01:11:17.239058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 65/131] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:16 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1266 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:17.261594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-1576ab45-502b-4599-a031-85727d2b18ac req-41692e95-27e8-4068-9e54-03e5dce635dd service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:17.261594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-1576ab45-502b-4599-a031-85727d2b18ac req-41692e95-27e8-4068-9e54-03e5dce635dd service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:17.261594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-1576ab45-502b-4599-a031-85727d2b18ac req-41692e95-27e8-4068-9e54-03e5dce635dd service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:17.285704 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-1576ab45-502b-4599-a031-85727d2b18ac req-41692e95-27e8-4068-9e54-03e5dce635dd service nova] Creating event network-changed:f71ffe87-943a-40bf-94f2-b17532b7092c for instance 96f9a9ce-f29e-48be-a36b-8518126a09ee on hv-cinder-maste Aug 07 01:11:17.293752 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-1576ab45-502b-4599-a031-85727d2b18ac req-41692e95-27e8-4068-9e54-03e5dce635dd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.085396 Aug 07 01:11:17.294872 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 67/132] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:11:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:11:18.268533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b96d717b-946b-42ab-bca5-245e4387a103 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:18.271116 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b96d717b-946b-42ab-bca5-245e4387a103 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:18.285996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b96d717b-946b-42ab-bca5-245e4387a103 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:18.288449 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b96d717b-946b-42ab-bca5-245e4387a103 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:18.289562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b96d717b-946b-42ab-bca5-245e4387a103 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:18.291920 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b96d717b-946b-42ab-bca5-245e4387a103 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:18.292912 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b96d717b-946b-42ab-bca5-245e4387a103 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:18.293693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b96d717b-946b-42ab-bca5-245e4387a103 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:18.431206 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b96d717b-946b-42ab-bca5-245e4387a103 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:18.451703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b96d717b-946b-42ab-bca5-245e4387a103 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1272 microversion: 2.1 time: 0.191797 Aug 07 01:11:18.468487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 66/133] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:18 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1272 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:19.531545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ca03e2fc-cc5b-493a-b7e9-469db52e9dc9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:19.536352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ca03e2fc-cc5b-493a-b7e9-469db52e9dc9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:19.561153 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca03e2fc-cc5b-493a-b7e9-469db52e9dc9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:19.562493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca03e2fc-cc5b-493a-b7e9-469db52e9dc9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:19.564077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca03e2fc-cc5b-493a-b7e9-469db52e9dc9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:19.566159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca03e2fc-cc5b-493a-b7e9-469db52e9dc9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:19.567550 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca03e2fc-cc5b-493a-b7e9-469db52e9dc9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:19.568669 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca03e2fc-cc5b-493a-b7e9-469db52e9dc9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:20.695533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-ef3e16fb-4cc4-4d13-8ce6-7f054a88591d req-4288faa6-22e6-471a-b082-f2f990e34135 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4f3f0774-3a34-4f19-9233-c4e55c7f5903", "tag": "9e5aa5e7-49cb-4c56-b9f5-3adf7c37f16a"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:20.756885 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-ef3e16fb-4cc4-4d13-8ce6-7f054a88591d req-4288faa6-22e6-471a-b082-f2f990e34135 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:20.758697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-ef3e16fb-4cc4-4d13-8ce6-7f054a88591d req-4288faa6-22e6-471a-b082-f2f990e34135 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:20.777535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-ef3e16fb-4cc4-4d13-8ce6-7f054a88591d req-4288faa6-22e6-471a-b082-f2f990e34135 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:20.869305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-ef3e16fb-4cc4-4d13-8ce6-7f054a88591d req-4288faa6-22e6-471a-b082-f2f990e34135 service nova] Creating event network-changed:9e5aa5e7-49cb-4c56-b9f5-3adf7c37f16a for instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 on hv-cinder-maste Aug 07 01:11:20.910181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-ef3e16fb-4cc4-4d13-8ce6-7f054a88591d req-4288faa6-22e6-471a-b082-f2f990e34135 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.217002 Aug 07 01:11:20.913816 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 67/134] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:11:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:11:20.934241 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6f3d7d0b-de5c-42c5-ab9f-da2c9aed0377 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:20.941290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6f3d7d0b-de5c-42c5-ab9f-da2c9aed0377 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:20.969720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f3d7d0b-de5c-42c5-ab9f-da2c9aed0377 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:20.970841 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f3d7d0b-de5c-42c5-ab9f-da2c9aed0377 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:20.973507 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f3d7d0b-de5c-42c5-ab9f-da2c9aed0377 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:20.981587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f3d7d0b-de5c-42c5-ab9f-da2c9aed0377 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:20.983054 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f3d7d0b-de5c-42c5-ab9f-da2c9aed0377 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:20.984079 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f3d7d0b-de5c-42c5-ab9f-da2c9aed0377 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:21.090624 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ca03e2fc-cc5b-493a-b7e9-469db52e9dc9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:21.136487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ca03e2fc-cc5b-493a-b7e9-469db52e9dc9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1314 microversion: 2.1 time: 1.632195 Aug 07 01:11:21.140057 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 68/135] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:19 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1314 bytes in 1638 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:21.176549 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6f3d7d0b-de5c-42c5-ab9f-da2c9aed0377 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Going to try to terminate instance {{(pid=117983) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:11:21.449247 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6f3d7d0b-de5c-42c5-ab9f-da2c9aed0377 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "DELETE /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 204 len: 0 microversion: 2.1 time: 0.530344 Aug 07 01:11:21.450507 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 68/136] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:11:20 2022] DELETE /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 07 01:11:21.466171 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4ef1c17c-6d80-433d-a42f-94dc3ae97592 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:21.470466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4ef1c17c-6d80-433d-a42f-94dc3ae97592 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:21.503450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ef1c17c-6d80-433d-a42f-94dc3ae97592 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:21.504740 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ef1c17c-6d80-433d-a42f-94dc3ae97592 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:21.511102 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ef1c17c-6d80-433d-a42f-94dc3ae97592 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:21.517202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ef1c17c-6d80-433d-a42f-94dc3ae97592 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:21.519804 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ef1c17c-6d80-433d-a42f-94dc3ae97592 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:21.524374 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ef1c17c-6d80-433d-a42f-94dc3ae97592 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:21.827150 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4ef1c17c-6d80-433d-a42f-94dc3ae97592 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:21.844983 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4ef1c17c-6d80-433d-a42f-94dc3ae97592 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1553 microversion: 2.1 time: 0.388124 Aug 07 01:11:21.848309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 69/137] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:21 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1553 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:22.179253 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-98182ef9-913e-4dbd-9026-47baefd48544 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:22.180190 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-98182ef9-913e-4dbd-9026-47baefd48544 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:22.195798 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98182ef9-913e-4dbd-9026-47baefd48544 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:22.197299 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98182ef9-913e-4dbd-9026-47baefd48544 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:22.199612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98182ef9-913e-4dbd-9026-47baefd48544 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:22.202209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98182ef9-913e-4dbd-9026-47baefd48544 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:22.203416 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98182ef9-913e-4dbd-9026-47baefd48544 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:22.210174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98182ef9-913e-4dbd-9026-47baefd48544 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:22.287916 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-f84f2ee8-9609-4c77-bf47-e97ba1ba3608 req-b7621dd3-f751-4ca4-a7f5-b9acfca18e44 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "202e0407-ce7c-4278-98b4-dd5855ee9088", "tag": "d572d867-1ee7-4d54-aac5-7ca50a5652f6"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:22.314329 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-f84f2ee8-9609-4c77-bf47-e97ba1ba3608 req-b7621dd3-f751-4ca4-a7f5-b9acfca18e44 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:22.315226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-f84f2ee8-9609-4c77-bf47-e97ba1ba3608 req-b7621dd3-f751-4ca4-a7f5-b9acfca18e44 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:22.316104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-f84f2ee8-9609-4c77-bf47-e97ba1ba3608 req-b7621dd3-f751-4ca4-a7f5-b9acfca18e44 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:22.341979 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-f84f2ee8-9609-4c77-bf47-e97ba1ba3608 req-b7621dd3-f751-4ca4-a7f5-b9acfca18e44 service nova] Creating event network-changed:d572d867-1ee7-4d54-aac5-7ca50a5652f6 for instance 202e0407-ce7c-4278-98b4-dd5855ee9088 on hv-cinder-maste Aug 07 01:11:22.353697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-f84f2ee8-9609-4c77-bf47-e97ba1ba3608 req-b7621dd3-f751-4ca4-a7f5-b9acfca18e44 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.082752 Aug 07 01:11:22.355201 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 70/138] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:11:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:11:22.648189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-98182ef9-913e-4dbd-9026-47baefd48544 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:22.665059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-98182ef9-913e-4dbd-9026-47baefd48544 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1314 microversion: 2.1 time: 0.492599 Aug 07 01:11:22.669135 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 69/139] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:22 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1314 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:22.891650 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5f4d8fb5-da65-440d-9c9d-02e83a1dd38d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:22.895418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5f4d8fb5-da65-440d-9c9d-02e83a1dd38d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:22.910544 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f4d8fb5-da65-440d-9c9d-02e83a1dd38d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:22.911839 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f4d8fb5-da65-440d-9c9d-02e83a1dd38d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:22.913268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f4d8fb5-da65-440d-9c9d-02e83a1dd38d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:22.916543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f4d8fb5-da65-440d-9c9d-02e83a1dd38d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:22.929893 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f4d8fb5-da65-440d-9c9d-02e83a1dd38d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:22.936545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f4d8fb5-da65-440d-9c9d-02e83a1dd38d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:23.474512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5f4d8fb5-da65-440d-9c9d-02e83a1dd38d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:23.507978 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5f4d8fb5-da65-440d-9c9d-02e83a1dd38d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1553 microversion: 2.1 time: 0.629906 Aug 07 01:11:23.509221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 71/140] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:22 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1553 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:23.711214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-513036e5-939b-456c-809b-274aa4168426 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:23.712852 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-513036e5-939b-456c-809b-274aa4168426 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:23.731265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-513036e5-939b-456c-809b-274aa4168426 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:23.732380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-513036e5-939b-456c-809b-274aa4168426 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:23.733462 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-513036e5-939b-456c-809b-274aa4168426 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:23.739295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-513036e5-939b-456c-809b-274aa4168426 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:23.741011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-513036e5-939b-456c-809b-274aa4168426 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:23.757355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-513036e5-939b-456c-809b-274aa4168426 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:24.550754 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9d8b78db-db3d-40db-8d2a-0733b79b364c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:24.554127 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9d8b78db-db3d-40db-8d2a-0733b79b364c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:24.568054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=60 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:24.572282 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9d8b78db-db3d-40db-8d2a-0733b79b364c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:24.573249 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9d8b78db-db3d-40db-8d2a-0733b79b364c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:24.574009 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9d8b78db-db3d-40db-8d2a-0733b79b364c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:24.575257 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9d8b78db-db3d-40db-8d2a-0733b79b364c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:24.576108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9d8b78db-db3d-40db-8d2a-0733b79b364c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:24.576856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9d8b78db-db3d-40db-8d2a-0733b79b364c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:25.099764 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:25.141570 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=6 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:25.238257 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9d8b78db-db3d-40db-8d2a-0733b79b364c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:25.275764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9d8b78db-db3d-40db-8d2a-0733b79b364c tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1553 microversion: 2.1 time: 0.733381 Aug 07 01:11:25.278342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 72/141] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:24 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1553 bytes in 738 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:25.338160 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-513036e5-939b-456c-809b-274aa4168426 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:25.355667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-513036e5-939b-456c-809b-274aa4168426 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1314 microversion: 2.1 time: 1.655064 Aug 07 01:11:25.370047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 70/142] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:23 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1314 bytes in 1671 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:26.003724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-c61fa26d-682e-4488-9643-7defe993463e req-74e4dca3-a9bc-48d6-a234-5c56de6e0354 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "93936488-825c-4db2-b648-ac856631c434", "name": "network-vif-deleted", "tag": "a2435539-4100-4730-a912-3afae7d39093"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:26.026108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:26.030921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c61fa26d-682e-4488-9643-7defe993463e req-74e4dca3-a9bc-48d6-a234-5c56de6e0354 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:26.032180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c61fa26d-682e-4488-9643-7defe993463e req-74e4dca3-a9bc-48d6-a234-5c56de6e0354 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:26.033199 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c61fa26d-682e-4488-9643-7defe993463e req-74e4dca3-a9bc-48d6-a234-5c56de6e0354 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:26.060049 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-c61fa26d-682e-4488-9643-7defe993463e req-74e4dca3-a9bc-48d6-a234-5c56de6e0354 service nova] Creating event network-vif-deleted:a2435539-4100-4730-a912-3afae7d39093 for instance 93936488-825c-4db2-b648-ac856631c434 on hv-cinder-maste Aug 07 01:11:26.071101 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-c61fa26d-682e-4488-9643-7defe993463e req-74e4dca3-a9bc-48d6-a234-5c56de6e0354 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074899 Aug 07 01:11:26.072876 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 73/143] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:11:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:11:26.322891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b9fe2584-3d32-4c1c-8f76-56995f6fde83 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:26.325470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b9fe2584-3d32-4c1c-8f76-56995f6fde83 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:26.340494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9fe2584-3d32-4c1c-8f76-56995f6fde83 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:26.342720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9fe2584-3d32-4c1c-8f76-56995f6fde83 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:26.343798 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9fe2584-3d32-4c1c-8f76-56995f6fde83 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:26.345876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9fe2584-3d32-4c1c-8f76-56995f6fde83 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:26.347745 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9fe2584-3d32-4c1c-8f76-56995f6fde83 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:26.348753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9fe2584-3d32-4c1c-8f76-56995f6fde83 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:26.396598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:26.417154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5d7c78a2-7abe-470f-a6b7-73919218787e tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:26.421522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5d7c78a2-7abe-470f-a6b7-73919218787e tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:26.440213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d7c78a2-7abe-470f-a6b7-73919218787e tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:26.441353 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d7c78a2-7abe-470f-a6b7-73919218787e tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:26.442673 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d7c78a2-7abe-470f-a6b7-73919218787e tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:26.444308 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d7c78a2-7abe-470f-a6b7-73919218787e tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:26.445401 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d7c78a2-7abe-470f-a6b7-73919218787e tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:26.446403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d7c78a2-7abe-470f-a6b7-73919218787e tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:26.497885 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b9fe2584-3d32-4c1c-8f76-56995f6fde83 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'caafb1ccfd704e3ca65b6c4db04b05ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a105c95deb1041ba9ac721d7c82110e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:26.519466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b9fe2584-3d32-4c1c-8f76-56995f6fde83 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 200 len: 1345 microversion: 2.1 time: 0.205445 Aug 07 01:11:26.521281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 71/144] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:26 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 1345 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:26.762753 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5d7c78a2-7abe-470f-a6b7-73919218787e tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:26.782100 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5d7c78a2-7abe-470f-a6b7-73919218787e tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1314 microversion: 2.1 time: 0.371078 Aug 07 01:11:26.793842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 74/145] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:26 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1314 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:27.587531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-493da0d0-fc8d-4f8c-aa0a-d4f671587e0d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:27.591617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-493da0d0-fc8d-4f8c-aa0a-d4f671587e0d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:27.631701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-493da0d0-fc8d-4f8c-aa0a-d4f671587e0d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:27.637617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-493da0d0-fc8d-4f8c-aa0a-d4f671587e0d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:27.638824 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-493da0d0-fc8d-4f8c-aa0a-d4f671587e0d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:27.640536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-493da0d0-fc8d-4f8c-aa0a-d4f671587e0d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:27.643598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-493da0d0-fc8d-4f8c-aa0a-d4f671587e0d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:27.644986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-493da0d0-fc8d-4f8c-aa0a-d4f671587e0d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:27.676941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-493da0d0-fc8d-4f8c-aa0a-d4f671587e0d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] HTTP exception thrown: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-493da0d0-fc8d-4f8c-aa0a-d4f671587e0d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Request method failure captured: Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 HTTP/1.1 Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xD_b2YBNQy0DxVQY_X_ZklYBKA5FYo9mXWVvH-CrdhRWocMwrdYXaECXTBslh7O873spfnNxME8h-ZszRv-lXvjCWj0xNK9SGNq5lVYUmiCyrmFCb7vvSlY-p1-gvtXlG0jo1yVqrzgiWrp_yuadks6-nv-XrREbH7Xxmy-zMXYRus Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: a105c95deb1041ba9ac721d7c82110e9 Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-DeleteServersTestJSON-144642853 Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:11:27.687886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:11:27.690677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:11:27.690677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-DeleteServersTestJSON-144642853 Aug 07 01:11:27.690677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: a105c95deb1041ba9ac721d7c82110e9 Aug 07 01:11:27.690677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-DeleteServersTestJSON-144642853 Aug 07 01:11:27.690677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-DeleteServersTestJSON-144642853-project Aug 07 01:11:27.690677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:11:27.690677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:11:27.690677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: caafb1ccfd704e3ca65b6c4db04b05ef Aug 07 01:11:27.690677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-DeleteServersTestJSON-144642853-project Aug 07 01:11:27.690677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:11:27.690677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:27.690677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '93936488-825c-4db2-b648-ac856631c434'} Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:11:27.693856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:27.696371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:27.702749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-493da0d0-fc8d-4f8c-aa0a-d4f671587e0d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Returning 404 to user: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:11:27.702749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-493da0d0-fc8d-4f8c-aa0a-d4f671587e0d tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 404 len: 111 microversion: 2.1 time: 0.137939 Aug 07 01:11:27.702749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 72/146] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:27 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 111 bytes in 140 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:11:27.828901 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-64f5e25d-426f-48be-9267-89ad1d389556 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:27.832340 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-64f5e25d-426f-48be-9267-89ad1d389556 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:27.841430 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-64f5e25d-426f-48be-9267-89ad1d389556 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:27.842250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-64f5e25d-426f-48be-9267-89ad1d389556 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:27.842933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-64f5e25d-426f-48be-9267-89ad1d389556 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:27.844390 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-64f5e25d-426f-48be-9267-89ad1d389556 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:27.846280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-64f5e25d-426f-48be-9267-89ad1d389556 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:27.847678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-64f5e25d-426f-48be-9267-89ad1d389556 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:28.012444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0850a39c-6150-4dc3-b7e0-414e55ca8065 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:28.017318 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0850a39c-6150-4dc3-b7e0-414e55ca8065 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:28.028531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0850a39c-6150-4dc3-b7e0-414e55ca8065 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:28.029689 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0850a39c-6150-4dc3-b7e0-414e55ca8065 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:28.030729 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0850a39c-6150-4dc3-b7e0-414e55ca8065 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:28.032149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0850a39c-6150-4dc3-b7e0-414e55ca8065 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:28.033206 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0850a39c-6150-4dc3-b7e0-414e55ca8065 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:28.034241 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0850a39c-6150-4dc3-b7e0-414e55ca8065 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:28.060022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-0850a39c-6150-4dc3-b7e0-414e55ca8065 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] HTTP exception thrown: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0850a39c-6150-4dc3-b7e0-414e55ca8065 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Request method failure captured: Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434/os-volume_attachments HTTP/1.1 Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xD_b2YBNQy0DxVQY_X_ZklYBKA5FYo9mXWVvH-CrdhRWocMwrdYXaECXTBslh7O873spfnNxME8h-ZszRv-lXvjCWj0xNK9SGNq5lVYUmiCyrmFCb7vvSlY-p1-gvtXlG0jo1yVqrzgiWrp_yuadks6-nv-XrREbH7Xxmy-zMXYRus Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: a105c95deb1041ba9ac721d7c82110e9 Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-DeleteServersTestJSON-144642853 Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:11:28.063943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:11:28.066301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:11:28.066301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-DeleteServersTestJSON-144642853 Aug 07 01:11:28.066301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: a105c95deb1041ba9ac721d7c82110e9 Aug 07 01:11:28.066301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-DeleteServersTestJSON-144642853 Aug 07 01:11:28.066301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-DeleteServersTestJSON-144642853-project Aug 07 01:11:28.066301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:11:28.066301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:11:28.066301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: caafb1ccfd704e3ca65b6c4db04b05ef Aug 07 01:11:28.066301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-DeleteServersTestJSON-144642853-project Aug 07 01:11:28.066301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:11:28.066301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:28.066301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'server_id': '93936488-825c-4db2-b648-ac856631c434'} Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:11:28.067888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:28.073889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:28.076303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0850a39c-6150-4dc3-b7e0-414e55ca8065 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Returning 404 to user: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:11:28.076303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0850a39c-6150-4dc3-b7e0-414e55ca8065 tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.062297 Aug 07 01:11:28.076303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 73/147] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:11:28 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434/os-volume_attachments => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:11:28.114082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-64f5e25d-426f-48be-9267-89ad1d389556 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:28.123903 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-64f5e25d-426f-48be-9267-89ad1d389556 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1314 microversion: 2.1 time: 0.303390 Aug 07 01:11:28.124655 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 75/148] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:27 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1314 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:29.164509 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-88ae6b07-9274-4f67-b235-82c942c34836 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:29.165648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-88ae6b07-9274-4f67-b235-82c942c34836 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:29.178152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88ae6b07-9274-4f67-b235-82c942c34836 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:29.178939 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88ae6b07-9274-4f67-b235-82c942c34836 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:29.179700 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88ae6b07-9274-4f67-b235-82c942c34836 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:29.180978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88ae6b07-9274-4f67-b235-82c942c34836 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:29.181684 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88ae6b07-9274-4f67-b235-82c942c34836 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:29.182309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88ae6b07-9274-4f67-b235-82c942c34836 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:29.443996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-88ae6b07-9274-4f67-b235-82c942c34836 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:29.461356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-88ae6b07-9274-4f67-b235-82c942c34836 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1314 microversion: 2.1 time: 0.304121 Aug 07 01:11:29.462990 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 74/149] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:29 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1314 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:29.494265 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ed4aae9b-4695-4e8e-ac76-5f12dbb8938a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:29.495365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ed4aae9b-4695-4e8e-ac76-5f12dbb8938a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:29.506676 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ed4aae9b-4695-4e8e-ac76-5f12dbb8938a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:29.507622 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ed4aae9b-4695-4e8e-ac76-5f12dbb8938a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:29.509160 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ed4aae9b-4695-4e8e-ac76-5f12dbb8938a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:29.509753 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ed4aae9b-4695-4e8e-ac76-5f12dbb8938a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:29.510848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ed4aae9b-4695-4e8e-ac76-5f12dbb8938a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:29.511909 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ed4aae9b-4695-4e8e-ac76-5f12dbb8938a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:29.542087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-ed4aae9b-4695-4e8e-ac76-5f12dbb8938a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] HTTP exception thrown: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ed4aae9b-4695-4e8e-ac76-5f12dbb8938a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Request method failure captured: Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: DELETE /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 HTTP/1.1 Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xD_b2YBNQy0DxVQY_X_ZklYBKA5FYo9mXWVvH-CrdhRWocMwrdYXaECXTBslh7O873spfnNxME8h-ZszRv-lXvjCWj0xNK9SGNq5lVYUmiCyrmFCb7vvSlY-p1-gvtXlG0jo1yVqrzgiWrp_yuadks6-nv-XrREbH7Xxmy-zMXYRus Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: a105c95deb1041ba9ac721d7c82110e9 Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-DeleteServersTestJSON-144642853 Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:11:29.547529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:11:29.550408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:11:29.550408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-DeleteServersTestJSON-144642853 Aug 07 01:11:29.550408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: a105c95deb1041ba9ac721d7c82110e9 Aug 07 01:11:29.550408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-DeleteServersTestJSON-144642853 Aug 07 01:11:29.550408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-DeleteServersTestJSON-144642853-project Aug 07 01:11:29.550408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:11:29.550408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:11:29.550408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: caafb1ccfd704e3ca65b6c4db04b05ef Aug 07 01:11:29.550408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-DeleteServersTestJSON-144642853-project Aug 07 01:11:29.550408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:11:29.550408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:29.550408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '93936488-825c-4db2-b648-ac856631c434'} Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:11:29.551870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:29.554525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:11:29.557215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ed4aae9b-4695-4e8e-ac76-5f12dbb8938a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Returning 404 to user: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:11:29.557215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ed4aae9b-4695-4e8e-ac76-5f12dbb8938a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "DELETE /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 404 len: 111 microversion: 2.1 time: 0.062637 Aug 07 01:11:29.557215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 76/150] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:11:29 2022] DELETE /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:11:29.565381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4a055082-8353-4ec9-bc24-770f6284ad7a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:29.566729 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4a055082-8353-4ec9-bc24-770f6284ad7a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] [instance: 93936488-825c-4db2-b648-ac856631c434] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:29.577969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a055082-8353-4ec9-bc24-770f6284ad7a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:29.578903 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a055082-8353-4ec9-bc24-770f6284ad7a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:29.579745 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a055082-8353-4ec9-bc24-770f6284ad7a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:29.581741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a055082-8353-4ec9-bc24-770f6284ad7a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:29.582624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a055082-8353-4ec9-bc24-770f6284ad7a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:29.583638 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a055082-8353-4ec9-bc24-770f6284ad7a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:29.611011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-4a055082-8353-4ec9-bc24-770f6284ad7a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] HTTP exception thrown: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4a055082-8353-4ec9-bc24-770f6284ad7a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Request method failure captured: Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 HTTP/1.1 Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xD_b2YBNQy0DxVQY_X_ZklYBKA5FYo9mXWVvH-CrdhRWocMwrdYXaECXTBslh7O873spfnNxME8h-ZszRv-lXvjCWj0xNK9SGNq5lVYUmiCyrmFCb7vvSlY-p1-gvtXlG0jo1yVqrzgiWrp_yuadks6-nv-XrREbH7Xxmy-zMXYRus Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: a105c95deb1041ba9ac721d7c82110e9 Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-DeleteServersTestJSON-144642853 Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:11:29.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:11:29.616526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a105c95deb1041ba9ac721d7c82110e9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:11:29.616526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-DeleteServersTestJSON-144642853 Aug 07 01:11:29.616526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: a105c95deb1041ba9ac721d7c82110e9 Aug 07 01:11:29.616526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-DeleteServersTestJSON-144642853 Aug 07 01:11:29.616526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-DeleteServersTestJSON-144642853-project Aug 07 01:11:29.616526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:11:29.616526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:11:29.616526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: caafb1ccfd704e3ca65b6c4db04b05ef Aug 07 01:11:29.616526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-DeleteServersTestJSON-144642853-project Aug 07 01:11:29.616526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:11:29.616526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:29.616526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '93936488-825c-4db2-b648-ac856631c434'} Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:11:29.617704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. Aug 07 01:11:29.620945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:11:29.624884 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4a055082-8353-4ec9-bc24-770f6284ad7a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] Returning 404 to user: Instance 93936488-825c-4db2-b648-ac856631c434 could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:11:29.624884 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4a055082-8353-4ec9-bc24-770f6284ad7a tempest-DeleteServersTestJSON-144642853 tempest-DeleteServersTestJSON-144642853-project] 10.222.0.26 "GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434" status: 404 len: 111 microversion: 2.1 time: 0.059855 Aug 07 01:11:29.624884 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 75/151] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:29 2022] GET /compute/v2.1/servers/93936488-825c-4db2-b648-ac856631c434 => generated 111 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:11:30.505307 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d52bb8b3-0b8e-4c09-bc77-3124feeb9b8b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:30.507545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d52bb8b3-0b8e-4c09-bc77-3124feeb9b8b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:30.529270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d52bb8b3-0b8e-4c09-bc77-3124feeb9b8b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:30.529270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d52bb8b3-0b8e-4c09-bc77-3124feeb9b8b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:30.529270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d52bb8b3-0b8e-4c09-bc77-3124feeb9b8b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:30.529270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d52bb8b3-0b8e-4c09-bc77-3124feeb9b8b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:30.529842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d52bb8b3-0b8e-4c09-bc77-3124feeb9b8b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:30.529842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d52bb8b3-0b8e-4c09-bc77-3124feeb9b8b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:30.881685 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d52bb8b3-0b8e-4c09-bc77-3124feeb9b8b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:30.891982 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d52bb8b3-0b8e-4c09-bc77-3124feeb9b8b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1314 microversion: 2.1 time: 0.393477 Aug 07 01:11:30.892902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 77/152] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:30 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1314 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:31.712576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-4037cf8a-e8b9-4006-84a0-ed7e1af7011d req-3157637d-057f-468f-9d33-051fbe348709 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "202e0407-ce7c-4278-98b4-dd5855ee9088", "name": "network-vif-plugged", "status": "completed", "tag": "d572d867-1ee7-4d54-aac5-7ca50a5652f6"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:31.775153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-4037cf8a-e8b9-4006-84a0-ed7e1af7011d req-3157637d-057f-468f-9d33-051fbe348709 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:31.776482 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-4037cf8a-e8b9-4006-84a0-ed7e1af7011d req-3157637d-057f-468f-9d33-051fbe348709 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:31.777461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-4037cf8a-e8b9-4006-84a0-ed7e1af7011d req-3157637d-057f-468f-9d33-051fbe348709 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:31.823085 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-4037cf8a-e8b9-4006-84a0-ed7e1af7011d req-3157637d-057f-468f-9d33-051fbe348709 service nova] Creating event network-vif-plugged:d572d867-1ee7-4d54-aac5-7ca50a5652f6 for instance 202e0407-ce7c-4278-98b4-dd5855ee9088 on hv-cinder-maste Aug 07 01:11:31.852039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-4037cf8a-e8b9-4006-84a0-ed7e1af7011d req-3157637d-057f-468f-9d33-051fbe348709 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.180491 Aug 07 01:11:31.855678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 76/153] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:11:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:11:31.920913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1b7f40c6-a4f9-4082-95af-cb1f191f080c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:31.922061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1b7f40c6-a4f9-4082-95af-cb1f191f080c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:31.943684 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b7f40c6-a4f9-4082-95af-cb1f191f080c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:31.945411 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b7f40c6-a4f9-4082-95af-cb1f191f080c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:31.947036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b7f40c6-a4f9-4082-95af-cb1f191f080c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:31.948761 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b7f40c6-a4f9-4082-95af-cb1f191f080c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:31.949896 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b7f40c6-a4f9-4082-95af-cb1f191f080c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:31.950938 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b7f40c6-a4f9-4082-95af-cb1f191f080c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:32.392771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1b7f40c6-a4f9-4082-95af-cb1f191f080c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:32.411887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1b7f40c6-a4f9-4082-95af-cb1f191f080c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1314 microversion: 2.1 time: 0.499387 Aug 07 01:11:32.413780 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 78/154] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:31 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1314 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:33.438177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1c924dd0-a61a-4549-bce2-d623edf53ca9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:33.440377 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1c924dd0-a61a-4549-bce2-d623edf53ca9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:33.457829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c924dd0-a61a-4549-bce2-d623edf53ca9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:33.464266 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c924dd0-a61a-4549-bce2-d623edf53ca9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:33.465756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c924dd0-a61a-4549-bce2-d623edf53ca9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:33.471124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c924dd0-a61a-4549-bce2-d623edf53ca9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:33.475768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c924dd0-a61a-4549-bce2-d623edf53ca9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:33.479513 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c924dd0-a61a-4549-bce2-d623edf53ca9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:33.852639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1c924dd0-a61a-4549-bce2-d623edf53ca9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:33.878496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1c924dd0-a61a-4549-bce2-d623edf53ca9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1314 microversion: 2.1 time: 0.444428 Aug 07 01:11:33.883026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 77/155] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:33 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1314 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:34.923203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-125454c8-6854-4782-91bc-90bec86cb5ff tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:34.924081 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-125454c8-6854-4782-91bc-90bec86cb5ff tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:34.943079 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-125454c8-6854-4782-91bc-90bec86cb5ff tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:34.945695 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-125454c8-6854-4782-91bc-90bec86cb5ff tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:34.946354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-125454c8-6854-4782-91bc-90bec86cb5ff tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:34.955674 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-125454c8-6854-4782-91bc-90bec86cb5ff tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:34.960417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-125454c8-6854-4782-91bc-90bec86cb5ff tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:34.963422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-125454c8-6854-4782-91bc-90bec86cb5ff tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:35.292380 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-125454c8-6854-4782-91bc-90bec86cb5ff tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:35.311660 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-125454c8-6854-4782-91bc-90bec86cb5ff tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1314 microversion: 2.1 time: 0.395177 Aug 07 01:11:35.313809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 79/156] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:34 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1314 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:36.366929 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ea0919d0-7d67-4e91-88b3-515554cdeb21 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:36.368134 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ea0919d0-7d67-4e91-88b3-515554cdeb21 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:36.395680 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ea0919d0-7d67-4e91-88b3-515554cdeb21 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:36.404830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ea0919d0-7d67-4e91-88b3-515554cdeb21 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:36.405752 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ea0919d0-7d67-4e91-88b3-515554cdeb21 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:36.407061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ea0919d0-7d67-4e91-88b3-515554cdeb21 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:36.408650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ea0919d0-7d67-4e91-88b3-515554cdeb21 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:36.409736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ea0919d0-7d67-4e91-88b3-515554cdeb21 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:36.985909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ea0919d0-7d67-4e91-88b3-515554cdeb21 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:37.014634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ea0919d0-7d67-4e91-88b3-515554cdeb21 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1314 microversion: 2.1 time: 0.664460 Aug 07 01:11:37.018945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 78/157] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:36 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1314 bytes in 671 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:38.053463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ef00e415-efc9-45f6-8d4a-beee783187e0 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:38.055189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ef00e415-efc9-45f6-8d4a-beee783187e0 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:38.066617 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef00e415-efc9-45f6-8d4a-beee783187e0 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:38.067456 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef00e415-efc9-45f6-8d4a-beee783187e0 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:38.068366 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef00e415-efc9-45f6-8d4a-beee783187e0 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:38.069257 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef00e415-efc9-45f6-8d4a-beee783187e0 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:38.069891 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef00e415-efc9-45f6-8d4a-beee783187e0 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:38.071594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef00e415-efc9-45f6-8d4a-beee783187e0 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:38.294467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ef00e415-efc9-45f6-8d4a-beee783187e0 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:38.305767 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ef00e415-efc9-45f6-8d4a-beee783187e0 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1499 microversion: 2.1 time: 0.267019 Aug 07 01:11:38.320683 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 80/158] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:38 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1499 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:38.993161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-48512587-9517-4436-8e64-61356d444d23 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:38.993997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-48512587-9517-4436-8e64-61356d444d23 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:39.005954 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-48512587-9517-4436-8e64-61356d444d23 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:39.007172 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-48512587-9517-4436-8e64-61356d444d23 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:39.007927 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-48512587-9517-4436-8e64-61356d444d23 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:39.009539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-48512587-9517-4436-8e64-61356d444d23 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:39.013902 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-48512587-9517-4436-8e64-61356d444d23 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:39.013902 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-48512587-9517-4436-8e64-61356d444d23 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:39.059448 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-48512587-9517-4436-8e64-61356d444d23 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Going to try to terminate instance {{(pid=117983) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:11:39.262697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-48512587-9517-4436-8e64-61356d444d23 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "DELETE /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 204 len: 0 microversion: 2.1 time: 0.277230 Aug 07 01:11:39.263523 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 79/159] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:11:38 2022] DELETE /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 07 01:11:39.279781 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-78b2d706-4c56-4d7f-87bf-a06caddd80dd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:39.281913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-78b2d706-4c56-4d7f-87bf-a06caddd80dd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:39.294877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-78b2d706-4c56-4d7f-87bf-a06caddd80dd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:39.296213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-78b2d706-4c56-4d7f-87bf-a06caddd80dd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:39.297250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-78b2d706-4c56-4d7f-87bf-a06caddd80dd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:39.299566 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-78b2d706-4c56-4d7f-87bf-a06caddd80dd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:39.301417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-78b2d706-4c56-4d7f-87bf-a06caddd80dd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:39.302397 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-78b2d706-4c56-4d7f-87bf-a06caddd80dd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:39.312370 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=94 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:39.523755 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-78b2d706-4c56-4d7f-87bf-a06caddd80dd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:39.540725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-78b2d706-4c56-4d7f-87bf-a06caddd80dd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1505 microversion: 2.1 time: 0.269180 Aug 07 01:11:39.542013 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 81/160] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:39 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1505 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:40.590979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7bb47aab-795a-48d3-b6cf-f22af5f3d3c5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:40.593379 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7bb47aab-795a-48d3-b6cf-f22af5f3d3c5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:40.608677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bb47aab-795a-48d3-b6cf-f22af5f3d3c5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:40.609678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bb47aab-795a-48d3-b6cf-f22af5f3d3c5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:40.610447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bb47aab-795a-48d3-b6cf-f22af5f3d3c5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:40.612846 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bb47aab-795a-48d3-b6cf-f22af5f3d3c5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:40.614434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bb47aab-795a-48d3-b6cf-f22af5f3d3c5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:40.615538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bb47aab-795a-48d3-b6cf-f22af5f3d3c5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:41.163519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7bb47aab-795a-48d3-b6cf-f22af5f3d3c5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:41.192334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7bb47aab-795a-48d3-b6cf-f22af5f3d3c5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1505 microversion: 2.1 time: 0.621414 Aug 07 01:11:41.212302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 80/161] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:40 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1505 bytes in 628 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:42.243101 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cd653cc4-22e8-49a9-b135-d2d1a1997567 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:42.245987 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-cd653cc4-22e8-49a9-b135-d2d1a1997567 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:42.271964 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd653cc4-22e8-49a9-b135-d2d1a1997567 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:42.274793 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd653cc4-22e8-49a9-b135-d2d1a1997567 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:42.275772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd653cc4-22e8-49a9-b135-d2d1a1997567 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:42.279046 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd653cc4-22e8-49a9-b135-d2d1a1997567 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:42.281051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd653cc4-22e8-49a9-b135-d2d1a1997567 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:42.285326 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd653cc4-22e8-49a9-b135-d2d1a1997567 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:42.394035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-a821b6e7-6e4f-4655-af61-908ba6ff015e req-64aae183-d164-4c18-9d44-4ab0b06e4e68 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "202e0407-ce7c-4278-98b4-dd5855ee9088", "name": "network-vif-deleted", "tag": "d572d867-1ee7-4d54-aac5-7ca50a5652f6"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:42.468760 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-cd653cc4-22e8-49a9-b135-d2d1a1997567 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:42.473284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a821b6e7-6e4f-4655-af61-908ba6ff015e req-64aae183-d164-4c18-9d44-4ab0b06e4e68 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:42.479982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a821b6e7-6e4f-4655-af61-908ba6ff015e req-64aae183-d164-4c18-9d44-4ab0b06e4e68 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:42.481348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a821b6e7-6e4f-4655-af61-908ba6ff015e req-64aae183-d164-4c18-9d44-4ab0b06e4e68 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:42.512836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-a821b6e7-6e4f-4655-af61-908ba6ff015e req-64aae183-d164-4c18-9d44-4ab0b06e4e68 service nova] Creating event network-vif-deleted:d572d867-1ee7-4d54-aac5-7ca50a5652f6 for instance 202e0407-ce7c-4278-98b4-dd5855ee9088 on hv-cinder-maste Aug 07 01:11:42.521721 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-cd653cc4-22e8-49a9-b135-d2d1a1997567 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 200 len: 1463 microversion: 2.1 time: 0.284975 Aug 07 01:11:42.522152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-a821b6e7-6e4f-4655-af61-908ba6ff015e req-64aae183-d164-4c18-9d44-4ab0b06e4e68 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.135036 Aug 07 01:11:42.523459 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 82/162] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:42 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 1463 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:42.543447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 81/163] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:11:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:11:43.563448 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4127878a-eb9f-43f4-98a1-059c0a9219a4 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:43.564442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4127878a-eb9f-43f4-98a1-059c0a9219a4 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 202e0407-ce7c-4278-98b4-dd5855ee9088] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:43.592458 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4127878a-eb9f-43f4-98a1-059c0a9219a4 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:43.593965 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4127878a-eb9f-43f4-98a1-059c0a9219a4 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:43.594992 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4127878a-eb9f-43f4-98a1-059c0a9219a4 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:43.597223 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4127878a-eb9f-43f4-98a1-059c0a9219a4 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:43.598374 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4127878a-eb9f-43f4-98a1-059c0a9219a4 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:43.600141 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4127878a-eb9f-43f4-98a1-059c0a9219a4 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:43.646105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-4127878a-eb9f-43f4-98a1-059c0a9219a4 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] HTTP exception thrown: Instance 202e0407-ce7c-4278-98b4-dd5855ee9088 could not be found. Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4127878a-eb9f-43f4-98a1-059c0a9219a4 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Request method failure captured: Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 HTTP/1.1 Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xEvnCKhv_-6LE6Um031urOZQaAAt-YamYCb7HhKLY-mIhVKVo1ZjqLKAq6gmY3LtPMj9CUh7oZEpIH7ELdw9Vb0bAx5P3NnhUBTMcy1-w4OCPMLhtRXUP_3Mh8dOWviq83wPHfFsxdB4XdDahM2XmCR8uFR4Z0dBTeqXTQbjVpMer0 Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 1c3de7a94e61473aa759f03c19119775 Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-VolumesActionsTest-556910903 Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:11:43.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:11:43.657631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c3de7a94e61473aa759f03c19119775"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1c3de7a94e61473aa759f03c19119775"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1c3de7a94e61473aa759f03c19119775"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c3de7a94e61473aa759f03c19119775"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:11:43.657631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-VolumesActionsTest-556910903 Aug 07 01:11:43.657631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 1c3de7a94e61473aa759f03c19119775 Aug 07 01:11:43.657631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-VolumesActionsTest-556910903 Aug 07 01:11:43.657631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-VolumesActionsTest-556910903-project Aug 07 01:11:43.657631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:11:43.657631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:11:43.657631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: dfc9712739724c91bd9dd5fdaaebd642 Aug 07 01:11:43.657631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-VolumesActionsTest-556910903-project Aug 07 01:11:43.657631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:11:43.657631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 202e0407-ce7c-4278-98b4-dd5855ee9088 could not be found. Aug 07 01:11:43.657631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '202e0407-ce7c-4278-98b4-dd5855ee9088'} Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 202e0407-ce7c-4278-98b4-dd5855ee9088 could not be found. Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:11:43.659664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 202e0407-ce7c-4278-98b4-dd5855ee9088 could not be found. Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 202e0407-ce7c-4278-98b4-dd5855ee9088 could not be found. Aug 07 01:11:43.663022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:11:43.667046 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4127878a-eb9f-43f4-98a1-059c0a9219a4 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Returning 404 to user: Instance 202e0407-ce7c-4278-98b4-dd5855ee9088 could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:11:43.667046 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4127878a-eb9f-43f4-98a1-059c0a9219a4 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088" status: 404 len: 111 microversion: 2.1 time: 0.106194 Aug 07 01:11:43.668054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 83/164] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:43 2022] GET /compute/v2.1/servers/202e0407-ce7c-4278-98b4-dd5855ee9088 => generated 111 bytes in 114 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:11:44.009459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1330784825", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "networks": [{"uuid": "c801bdeb-0c37-4d5c-9982-2308bc5204c7"}]}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:44.199740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.network.neutron [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] validate_networks() for [('c801bdeb-0c37-4d5c-9982-2308bc5204c7', None, None, None, None, None)] {{(pid=117983) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:11:45.564870 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Flavor limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:11:45.566343 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Image limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:11:45.566864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Flavor pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:11:45.567397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Image pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:11:45.568288 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:11:45.595520 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:45.596755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:45.597716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:45.599595 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:45.601069 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:45.602145 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:45.637658 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Getting quotas for project 1c3de7a94e61473aa759f03c19119775. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:11:45.645470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Getting quotas for user dfc9712739724c91bd9dd5fdaaebd642 and project 1c3de7a94e61473aa759f03c19119775. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:11:45.672806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Going to run 1 instances... {{(pid=117983) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:11:45.678697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117983) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:11:45.776760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-49feedd6-dea2-4951-8e02-e5f22c702281 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.790584 Aug 07 01:11:45.784286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 82/165] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:11:43 2022] POST /compute/v2.1/servers => generated 380 bytes in 1795 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:11:45.797966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bca3b03c-3d47-4290-9105-7c0557900d1e tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:45.800705 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bca3b03c-3d47-4290-9105-7c0557900d1e tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:46.217965 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bca3b03c-3d47-4290-9105-7c0557900d1e tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:46.254780 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bca3b03c-3d47-4290-9105-7c0557900d1e tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1214 microversion: 2.1 time: 0.467829 Aug 07 01:11:46.269971 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 84/166] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:45 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1214 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:47.294046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0af59580-0b1d-4107-a1d8-9675c8f53ddd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:47.294892 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0af59580-0b1d-4107-a1d8-9675c8f53ddd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:47.316474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=57,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:47.323646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0af59580-0b1d-4107-a1d8-9675c8f53ddd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:47.326174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0af59580-0b1d-4107-a1d8-9675c8f53ddd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:47.327126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0af59580-0b1d-4107-a1d8-9675c8f53ddd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:47.342384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0af59580-0b1d-4107-a1d8-9675c8f53ddd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:47.343788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0af59580-0b1d-4107-a1d8-9675c8f53ddd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:47.344716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0af59580-0b1d-4107-a1d8-9675c8f53ddd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:47.528121 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0af59580-0b1d-4107-a1d8-9675c8f53ddd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:47.566323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0af59580-0b1d-4107-a1d8-9675c8f53ddd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1210 microversion: 2.1 time: 0.273136 Aug 07 01:11:47.582138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 83/167] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:47 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1210 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:48.630464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-903ac15c-a09d-446e-9fee-91fa1907af17 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:48.636542 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-903ac15c-a09d-446e-9fee-91fa1907af17 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:48.685489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-903ac15c-a09d-446e-9fee-91fa1907af17 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:48.687692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-903ac15c-a09d-446e-9fee-91fa1907af17 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:48.691868 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-903ac15c-a09d-446e-9fee-91fa1907af17 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:48.695639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-903ac15c-a09d-446e-9fee-91fa1907af17 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:48.697860 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-903ac15c-a09d-446e-9fee-91fa1907af17 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:48.699146 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-903ac15c-a09d-446e-9fee-91fa1907af17 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:49.054249 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-903ac15c-a09d-446e-9fee-91fa1907af17 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:49.086837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-903ac15c-a09d-446e-9fee-91fa1907af17 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1272 microversion: 2.1 time: 0.470218 Aug 07 01:11:49.091490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 85/168] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:48 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1272 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:50.116865 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2d693f95-360c-41b8-ba80-975261ce427a tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:50.124408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2d693f95-360c-41b8-ba80-975261ce427a tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:50.162897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d693f95-360c-41b8-ba80-975261ce427a tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:50.166513 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d693f95-360c-41b8-ba80-975261ce427a tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:50.171755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d693f95-360c-41b8-ba80-975261ce427a tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:50.173337 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d693f95-360c-41b8-ba80-975261ce427a tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:50.177809 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d693f95-360c-41b8-ba80-975261ce427a tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:50.178741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d693f95-360c-41b8-ba80-975261ce427a tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:50.961352 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2d693f95-360c-41b8-ba80-975261ce427a tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:51.012994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2d693f95-360c-41b8-ba80-975261ce427a tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1314 microversion: 2.1 time: 0.904301 Aug 07 01:11:51.014040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 84/169] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:50 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1314 bytes in 907 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:52.064009 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cc7fd19d-78e7-4793-9acc-5d712860489c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:52.070806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-cc7fd19d-78e7-4793-9acc-5d712860489c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:52.097325 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc7fd19d-78e7-4793-9acc-5d712860489c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:52.099177 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc7fd19d-78e7-4793-9acc-5d712860489c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:52.103496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc7fd19d-78e7-4793-9acc-5d712860489c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:52.105526 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc7fd19d-78e7-4793-9acc-5d712860489c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:52.110373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc7fd19d-78e7-4793-9acc-5d712860489c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:52.112228 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc7fd19d-78e7-4793-9acc-5d712860489c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:52.681377 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-c5510a8c-325b-43d9-b396-9bd2d403d2ab req-5cee22c4-9e43-41a0-9411-9f0defb650b7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "95d224a0-7a33-422a-bae6-06f4acfac258", "tag": "0108d266-48bb-4aec-97ba-3cbd0446ef81"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:52.785677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-c5510a8c-325b-43d9-b396-9bd2d403d2ab req-5cee22c4-9e43-41a0-9411-9f0defb650b7 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:52.786820 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-c5510a8c-325b-43d9-b396-9bd2d403d2ab req-5cee22c4-9e43-41a0-9411-9f0defb650b7 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:52.787441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-c5510a8c-325b-43d9-b396-9bd2d403d2ab req-5cee22c4-9e43-41a0-9411-9f0defb650b7 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:52.874344 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-c5510a8c-325b-43d9-b396-9bd2d403d2ab req-5cee22c4-9e43-41a0-9411-9f0defb650b7 service nova] Creating event network-changed:0108d266-48bb-4aec-97ba-3cbd0446ef81 for instance 95d224a0-7a33-422a-bae6-06f4acfac258 on hv-cinder-maste Aug 07 01:11:52.886216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-c5510a8c-325b-43d9-b396-9bd2d403d2ab req-5cee22c4-9e43-41a0-9411-9f0defb650b7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.222599 Aug 07 01:11:52.887966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 85/170] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:11:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:11:52.938779 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8687ffa3-f501-4702-8e33-4dfefc162e55 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:52.941177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8687ffa3-f501-4702-8e33-4dfefc162e55 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:52.961412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8687ffa3-f501-4702-8e33-4dfefc162e55 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:52.962284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8687ffa3-f501-4702-8e33-4dfefc162e55 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:52.964572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8687ffa3-f501-4702-8e33-4dfefc162e55 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:52.965877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8687ffa3-f501-4702-8e33-4dfefc162e55 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:52.966973 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8687ffa3-f501-4702-8e33-4dfefc162e55 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:52.968277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8687ffa3-f501-4702-8e33-4dfefc162e55 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:53.037867 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8687ffa3-f501-4702-8e33-4dfefc162e55 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee/ips" status: 200 len: 150 microversion: 2.1 time: 0.145410 Aug 07 01:11:53.048297 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 86/171] 10.222.0.26 () {58 vars in 1248 bytes} [Sun Aug 7 01:11:52 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee/ips => generated 150 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:11:53.103725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-cc7fd19d-78e7-4793-9acc-5d712860489c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:53.184211 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-cc7fd19d-78e7-4793-9acc-5d712860489c tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1314 microversion: 2.1 time: 1.120141 Aug 07 01:11:53.184932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 86/172] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:52 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1314 bytes in 1130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:54.227036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8176d29e-4ce7-4385-914f-b09624031efa tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:54.230063 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8176d29e-4ce7-4385-914f-b09624031efa tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:54.255107 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8176d29e-4ce7-4385-914f-b09624031efa tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:54.256626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8176d29e-4ce7-4385-914f-b09624031efa tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:54.260017 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8176d29e-4ce7-4385-914f-b09624031efa tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:54.263633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8176d29e-4ce7-4385-914f-b09624031efa tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:54.266287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8176d29e-4ce7-4385-914f-b09624031efa tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:54.267586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8176d29e-4ce7-4385-914f-b09624031efa tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:54.725795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d9230cc7-d5f5-4c0f-8db8-eb33e4fd3982 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:54.728733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d9230cc7-d5f5-4c0f-8db8-eb33e4fd3982 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:54.744530 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=32 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:54.768093 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9230cc7-d5f5-4c0f-8db8-eb33e4fd3982 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:54.775685 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9230cc7-d5f5-4c0f-8db8-eb33e4fd3982 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:54.776611 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9230cc7-d5f5-4c0f-8db8-eb33e4fd3982 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:54.778950 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9230cc7-d5f5-4c0f-8db8-eb33e4fd3982 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:54.780023 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9230cc7-d5f5-4c0f-8db8-eb33e4fd3982 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:54.781668 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9230cc7-d5f5-4c0f-8db8-eb33e4fd3982 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:54.852031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d9230cc7-d5f5-4c0f-8db8-eb33e4fd3982 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/ips" status: 200 len: 146 microversion: 2.1 time: 0.146499 Aug 07 01:11:54.865044 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 87/173] 10.222.0.26 () {58 vars in 1248 bytes} [Sun Aug 7 01:11:54 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/ips => generated 146 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:11:55.097555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8176d29e-4ce7-4385-914f-b09624031efa tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:55.124366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8176d29e-4ce7-4385-914f-b09624031efa tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1314 microversion: 2.1 time: 0.899381 Aug 07 01:11:55.138364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 87/174] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:54 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1314 bytes in 919 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:55.418402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a2b5054f-6a2c-4d7d-9bff-82d1d9ae9e6c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "fa48cc70-a60b-440f-8b1a-09ffff661711"}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:55.421548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a2b5054f-6a2c-4d7d-9bff-82d1d9ae9e6c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:55.432279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a2b5054f-6a2c-4d7d-9bff-82d1d9ae9e6c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:55.433447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a2b5054f-6a2c-4d7d-9bff-82d1d9ae9e6c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:55.433967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a2b5054f-6a2c-4d7d-9bff-82d1d9ae9e6c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:55.434946 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a2b5054f-6a2c-4d7d-9bff-82d1d9ae9e6c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:55.435567 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a2b5054f-6a2c-4d7d-9bff-82d1d9ae9e6c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:55.436928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a2b5054f-6a2c-4d7d-9bff-82d1d9ae9e6c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:55.619549 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:55.642301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1829931638", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:55.663225 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=6 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:55.729299 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.network.neutron [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] validate_networks() for [('fd0a1aca-5011-496b-8c8b-5bafdabdf653', None, None, None, None, None)] {{(pid=117984) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:11:56.235260 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:56.317286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a2b5054f-6a2c-4d7d-9bff-82d1d9ae9e6c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "POST /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.901579 Aug 07 01:11:56.319556 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 88/175] 10.222.0.26 () {60 vars in 1323 bytes} [Sun Aug 7 01:11:55 2022] POST /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee/os-volume_attachments => generated 194 bytes in 906 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:11:56.374508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7c6f5cec-9c2f-4300-9a6e-e2b75cc1990e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:56.375526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7c6f5cec-9c2f-4300-9a6e-e2b75cc1990e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:56.394960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c6f5cec-9c2f-4300-9a6e-e2b75cc1990e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:56.396503 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c6f5cec-9c2f-4300-9a6e-e2b75cc1990e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:56.397345 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c6f5cec-9c2f-4300-9a6e-e2b75cc1990e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:56.398832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c6f5cec-9c2f-4300-9a6e-e2b75cc1990e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:56.399908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c6f5cec-9c2f-4300-9a6e-e2b75cc1990e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:56.401146 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c6f5cec-9c2f-4300-9a6e-e2b75cc1990e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:56.453698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:11:56.470624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7c6f5cec-9c2f-4300-9a6e-e2b75cc1990e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/ips" status: 200 len: 153 microversion: 2.20 time: 0.129710 Aug 07 01:11:56.472623 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 89/176] 10.222.0.26 () {60 vars in 1289 bytes} [Sun Aug 7 01:11:56 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/ips => generated 153 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:11:56.486640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8c7e7568-2c96-4da1-a3be-039c6f30f58f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:56.489101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8c7e7568-2c96-4da1-a3be-039c6f30f58f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:56.503724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c7e7568-2c96-4da1-a3be-039c6f30f58f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:56.505119 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c7e7568-2c96-4da1-a3be-039c6f30f58f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:56.506228 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c7e7568-2c96-4da1-a3be-039c6f30f58f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:56.507940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c7e7568-2c96-4da1-a3be-039c6f30f58f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:56.508571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c7e7568-2c96-4da1-a3be-039c6f30f58f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:56.509329 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c7e7568-2c96-4da1-a3be-039c6f30f58f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:57.205582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8c7e7568-2c96-4da1-a3be-039c6f30f58f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:57.258795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8c7e7568-2c96-4da1-a3be-039c6f30f58f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1314 microversion: 2.1 time: 0.781911 Aug 07 01:11:57.262301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 90/177] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:56 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1314 bytes in 789 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:11:57.983564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Flavor limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:11:57.985659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Image limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:11:57.990216 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Flavor pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:11:57.992299 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Image pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:11:57.993736 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:11:58.075516 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:58.078163 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:58.078924 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:58.080354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:58.084325 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:58.085046 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:58.121247 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Getting quotas for project 9badaa18dcf64810b10dd7bf03d8e40d. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:11:58.130492 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Getting quotas for user ec931c9cc56a462991413926d34dfa46 and project 9badaa18dcf64810b10dd7bf03d8e40d. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:11:58.179844 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Going to run 1 instances... {{(pid=117984) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:11:58.185407 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117984) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:11:58.306908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1f6fc0de-7303-42e5-be70-8d5a3db00359 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:58.308033 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1f6fc0de-7303-42e5-be70-8d5a3db00359 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:58.353490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1f6fc0de-7303-42e5-be70-8d5a3db00359 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:58.360377 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1f6fc0de-7303-42e5-be70-8d5a3db00359 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:58.367697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1f6fc0de-7303-42e5-be70-8d5a3db00359 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:58.375350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1f6fc0de-7303-42e5-be70-8d5a3db00359 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:58.376195 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1f6fc0de-7303-42e5-be70-8d5a3db00359 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:58.377031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1f6fc0de-7303-42e5-be70-8d5a3db00359 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:58.381684 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1f4c7a5a-976d-4f15-91c9-4382ccb18c0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 3.514228 Aug 07 01:11:58.391476 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 88/178] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:11:54 2022] POST /compute/v2.1/servers => generated 383 bytes in 3525 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:11:58.426737 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4319579f-3188-4309-a657-31237d82ae51 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:11:58.427454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4319579f-3188-4309-a657-31237d82ae51 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:58.570925 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4319579f-3188-4309-a657-31237d82ae51 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:58.585644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4319579f-3188-4309-a657-31237d82ae51 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1222 microversion: 2.1 time: 0.173284 Aug 07 01:11:58.586824 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 89/179] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:58 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1222 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:58.854271 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e74ba705-e0cb-4901-92f2-894e25714214 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:58.855064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e74ba705-e0cb-4901-92f2-894e25714214 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:58.872090 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e74ba705-e0cb-4901-92f2-894e25714214 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:58.873306 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e74ba705-e0cb-4901-92f2-894e25714214 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:58.873606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e74ba705-e0cb-4901-92f2-894e25714214 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:58.874536 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e74ba705-e0cb-4901-92f2-894e25714214 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:58.875127 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e74ba705-e0cb-4901-92f2-894e25714214 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:58.881237 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e74ba705-e0cb-4901-92f2-894e25714214 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:58.955028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.objects.instance [None req-e74ba705-e0cb-4901-92f2-894e25714214 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lazy-loading 'flavor' on Instance uuid 7f60fb22-d17a-4b7c-9b7d-95543be1907e {{(pid=117984) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:11:59.192033 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1f6fc0de-7303-42e5-be70-8d5a3db00359 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:11:59.226829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1f6fc0de-7303-42e5-be70-8d5a3db00359 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1314 microversion: 2.1 time: 0.930309 Aug 07 01:11:59.229469 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 91/180] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:11:58 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1314 bytes in 935 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:11:59.258139 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2049312973", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:59.436091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.network.neutron [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] validate_networks() for [('fd0a1aca-5011-496b-8c8b-5bafdabdf653', None, None, None, None, None)] {{(pid=117983) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:11:59.542908 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e74ba705-e0cb-4901-92f2-894e25714214 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "POST /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/action" status: 202 len: 0 microversion: 2.20 time: 0.689695 Aug 07 01:11:59.543998 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 90/181] 10.222.0.26 () {62 vars in 1319 bytes} [Sun Aug 7 01:11:58 2022] POST /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/action => generated 0 bytes in 695 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 07 01:11:59.555861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-86ec5dac-3373-4039-a13f-474591d424cd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c045b917-ed68-46a0-a932-c7b3fe80957f"}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:11:59.558570 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-86ec5dac-3373-4039-a13f-474591d424cd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:11:59.588447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86ec5dac-3373-4039-a13f-474591d424cd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:59.590204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86ec5dac-3373-4039-a13f-474591d424cd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:59.592027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86ec5dac-3373-4039-a13f-474591d424cd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:11:59.592924 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86ec5dac-3373-4039-a13f-474591d424cd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:11:59.608330 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86ec5dac-3373-4039-a13f-474591d424cd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:11:59.608945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86ec5dac-3373-4039-a13f-474591d424cd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:00.412525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-86ec5dac-3373-4039-a13f-474591d424cd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "POST /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.866206 Aug 07 01:12:00.416391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 91/182] 10.222.0.26 () {60 vars in 1323 bytes} [Sun Aug 7 01:11:59 2022] POST /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/os-volume_attachments => generated 194 bytes in 870 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:12:00.460600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-88d871b3-b9eb-43a9-be47-1a87de4494d9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:00.461966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-88d871b3-b9eb-43a9-be47-1a87de4494d9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:00.526830 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-88d871b3-b9eb-43a9-be47-1a87de4494d9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:00.527717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-88d871b3-b9eb-43a9-be47-1a87de4494d9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:00.528571 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-88d871b3-b9eb-43a9-be47-1a87de4494d9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:00.529863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-88d871b3-b9eb-43a9-be47-1a87de4494d9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:00.530723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-88d871b3-b9eb-43a9-be47-1a87de4494d9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:00.531473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-88d871b3-b9eb-43a9-be47-1a87de4494d9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:00.573502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Flavor limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:12:00.574244 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Image limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:12:00.575262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Flavor pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:12:00.576007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Image pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:12:00.576940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:12:00.626159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:00.626880 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:00.628271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:00.630539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:00.640635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:00.642469 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:00.715461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Getting quotas for project 9badaa18dcf64810b10dd7bf03d8e40d. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:12:00.731732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Getting quotas for user ec931c9cc56a462991413926d34dfa46 and project 9badaa18dcf64810b10dd7bf03d8e40d. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:12:00.785371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Going to run 1 instances... {{(pid=117983) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:12:00.788584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117983) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:12:00.968810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a9557025-2250-487e-9e2c-e7bdc74bdf26 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.736496 Aug 07 01:12:00.970222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 92/183] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:11:59 2022] POST /compute/v2.1/servers => generated 383 bytes in 1740 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:12:01.002981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d86f583e-f574-4c05-9041-f9aa71aebed9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:01.005836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d86f583e-f574-4c05-9041-f9aa71aebed9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:01.033434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d86f583e-f574-4c05-9041-f9aa71aebed9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:01.044519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d86f583e-f574-4c05-9041-f9aa71aebed9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:01.046656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d86f583e-f574-4c05-9041-f9aa71aebed9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:01.048686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d86f583e-f574-4c05-9041-f9aa71aebed9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:01.049734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d86f583e-f574-4c05-9041-f9aa71aebed9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:01.050660 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d86f583e-f574-4c05-9041-f9aa71aebed9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:01.055391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-88d871b3-b9eb-43a9-be47-1a87de4494d9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:01.122510 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-88d871b3-b9eb-43a9-be47-1a87de4494d9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:01.124936 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-88d871b3-b9eb-43a9-be47-1a87de4494d9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:01.134640 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-88d871b3-b9eb-43a9-be47-1a87de4494d9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.712332 Aug 07 01:12:01.144745 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 92/184] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:00 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 724 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:01.152106 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-874bd19a-4c06-473a-8033-2de96ba72ed0 req-009de436-7553-4ee5-8817-6864951f0134 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "95d224a0-7a33-422a-bae6-06f4acfac258", "name": "network-vif-plugged", "status": "completed", "tag": "0108d266-48bb-4aec-97ba-3cbd0446ef81"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:01.204790 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-874bd19a-4c06-473a-8033-2de96ba72ed0 req-009de436-7553-4ee5-8817-6864951f0134 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:01.206209 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-874bd19a-4c06-473a-8033-2de96ba72ed0 req-009de436-7553-4ee5-8817-6864951f0134 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:01.207031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-874bd19a-4c06-473a-8033-2de96ba72ed0 req-009de436-7553-4ee5-8817-6864951f0134 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:01.309390 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-874bd19a-4c06-473a-8033-2de96ba72ed0 req-009de436-7553-4ee5-8817-6864951f0134 service nova] Creating event network-vif-plugged:0108d266-48bb-4aec-97ba-3cbd0446ef81 for instance 95d224a0-7a33-422a-bae6-06f4acfac258 on hv-cinder-maste Aug 07 01:12:01.327556 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-874bd19a-4c06-473a-8033-2de96ba72ed0 req-009de436-7553-4ee5-8817-6864951f0134 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.180624 Aug 07 01:12:01.332401 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 93/185] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:12:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:12:01.357176 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b2ac55d1-2914-41d0-810d-d04b2d8b4a23 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:01.361824 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b2ac55d1-2914-41d0-810d-d04b2d8b4a23 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:01.535586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d86f583e-f574-4c05-9041-f9aa71aebed9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:01.551068 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d86f583e-f574-4c05-9041-f9aa71aebed9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1314 microversion: 2.1 time: 0.572777 Aug 07 01:12:01.564509 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 93/186] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:00 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1314 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:01.690432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b2ac55d1-2914-41d0-810d-d04b2d8b4a23 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:01.724478 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b2ac55d1-2914-41d0-810d-d04b2d8b4a23 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1222 microversion: 2.1 time: 0.387118 Aug 07 01:12:01.736510 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 94/187] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:01 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1222 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:02.175232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6058c764-f07d-4d1a-bfe1-61d810edccd4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:02.176253 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6058c764-f07d-4d1a-bfe1-61d810edccd4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:02.212640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6058c764-f07d-4d1a-bfe1-61d810edccd4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:02.217911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6058c764-f07d-4d1a-bfe1-61d810edccd4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:02.219219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6058c764-f07d-4d1a-bfe1-61d810edccd4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:02.221153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6058c764-f07d-4d1a-bfe1-61d810edccd4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:02.222857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6058c764-f07d-4d1a-bfe1-61d810edccd4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:02.223893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6058c764-f07d-4d1a-bfe1-61d810edccd4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:02.578841 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5088f993-5164-4f1a-9f96-465bc1e86db1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:02.581610 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5088f993-5164-4f1a-9f96-465bc1e86db1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:02.650355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5088f993-5164-4f1a-9f96-465bc1e86db1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:02.651264 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5088f993-5164-4f1a-9f96-465bc1e86db1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:02.652098 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5088f993-5164-4f1a-9f96-465bc1e86db1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:02.661337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5088f993-5164-4f1a-9f96-465bc1e86db1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:02.662881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5088f993-5164-4f1a-9f96-465bc1e86db1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:02.667563 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5088f993-5164-4f1a-9f96-465bc1e86db1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:03.226160 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6058c764-f07d-4d1a-bfe1-61d810edccd4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:03.248599 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6058c764-f07d-4d1a-bfe1-61d810edccd4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:03.253906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6058c764-f07d-4d1a-bfe1-61d810edccd4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:03.255648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6058c764-f07d-4d1a-bfe1-61d810edccd4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 1.087652 Aug 07 01:12:03.265019 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 94/188] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:02 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 1098 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:03.282079 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4b355bb1-e3ff-4596-9db9-b5193d52a824 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:03.282972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4b355bb1-e3ff-4596-9db9-b5193d52a824 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:03.296419 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b355bb1-e3ff-4596-9db9-b5193d52a824 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:03.297470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b355bb1-e3ff-4596-9db9-b5193d52a824 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:03.298109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b355bb1-e3ff-4596-9db9-b5193d52a824 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:03.299578 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b355bb1-e3ff-4596-9db9-b5193d52a824 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:03.300327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b355bb1-e3ff-4596-9db9-b5193d52a824 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:03.300934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b355bb1-e3ff-4596-9db9-b5193d52a824 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:03.479603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4b355bb1-e3ff-4596-9db9-b5193d52a824 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:03.515596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4b355bb1-e3ff-4596-9db9-b5193d52a824 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1218 microversion: 2.1 time: 0.247510 Aug 07 01:12:03.532518 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 95/189] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:03 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1218 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:03.559021 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5088f993-5164-4f1a-9f96-465bc1e86db1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:03.587571 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5088f993-5164-4f1a-9f96-465bc1e86db1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1314 microversion: 2.1 time: 1.013024 Aug 07 01:12:03.600195 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 95/190] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:02 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1314 bytes in 1027 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:04.311164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6219c77f-fb7b-4c89-b51b-fc047fbdea42 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:04.316084 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6219c77f-fb7b-4c89-b51b-fc047fbdea42 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:04.337637 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6219c77f-fb7b-4c89-b51b-fc047fbdea42 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:04.340219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6219c77f-fb7b-4c89-b51b-fc047fbdea42 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:04.341847 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6219c77f-fb7b-4c89-b51b-fc047fbdea42 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:04.343077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6219c77f-fb7b-4c89-b51b-fc047fbdea42 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:04.344889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6219c77f-fb7b-4c89-b51b-fc047fbdea42 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:04.346215 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6219c77f-fb7b-4c89-b51b-fc047fbdea42 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:04.427184 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fe280a23-15dc-4d87-8dfe-c69e796c2ff6 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Action: 'create', calling method: .version_select at 0x7f1fad533700>, body: {"keypair": {"name": "tempest-keypair-2078632662", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJwDK3U2a261Ggjd1Ne/ZpVG5ujCdDq6uw673+vTKqfnb1clhDIFQRXreMqwc2bwXdBqaqkK8vUqZFo8crvgNYm5dmRzjwTXJ1gz6zQYOx1IczYmHmyA5LGMX8OtHql5hQ=="}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:04.506407 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-fe280a23-15dc-4d87-8dfe-c69e796c2ff6 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for project 35820b4f79774ac49f43fce915f56d6f. Resources: {'key_pairs'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:12:04.544066 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-fe280a23-15dc-4d87-8dfe-c69e796c2ff6 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for user e65f0f1ba54f483abff788974b1256cb and project 35820b4f79774ac49f43fce915f56d6f. Resources: {'key_pairs'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:12:04.641687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fe280a23-15dc-4d87-8dfe-c69e796c2ff6 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.222703 Aug 07 01:12:04.645005 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 96/191] 10.222.0.26 () {60 vars in 1159 bytes} [Sun Aug 7 01:12:04 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:12:04.651798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-32ad3998-ecec-4a54-b64b-a7771a7881ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:04.653041 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-32ad3998-ecec-4a54-b64b-a7771a7881ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:04.665487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-32ad3998-ecec-4a54-b64b-a7771a7881ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:04.666450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-32ad3998-ecec-4a54-b64b-a7771a7881ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:04.667171 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-32ad3998-ecec-4a54-b64b-a7771a7881ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:04.668487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-32ad3998-ecec-4a54-b64b-a7771a7881ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:04.669409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-32ad3998-ecec-4a54-b64b-a7771a7881ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:04.670154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-32ad3998-ecec-4a54-b64b-a7771a7881ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:04.819001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6219c77f-fb7b-4c89-b51b-fc047fbdea42 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:04.839225 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6219c77f-fb7b-4c89-b51b-fc047fbdea42 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:04.843713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6219c77f-fb7b-4c89-b51b-fc047fbdea42 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:04.845221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6219c77f-fb7b-4c89-b51b-fc047fbdea42 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.539217 Aug 07 01:12:04.846898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 96/192] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:04 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:04.857841 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-81a265d1-0d04-47d7-8b5c-d49eb2724e81 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:04.859359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-81a265d1-0d04-47d7-8b5c-d49eb2724e81 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:04.872204 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81a265d1-0d04-47d7-8b5c-d49eb2724e81 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:04.874049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81a265d1-0d04-47d7-8b5c-d49eb2724e81 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:04.874892 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81a265d1-0d04-47d7-8b5c-d49eb2724e81 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:04.876148 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81a265d1-0d04-47d7-8b5c-d49eb2724e81 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:04.876716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81a265d1-0d04-47d7-8b5c-d49eb2724e81 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:04.877165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81a265d1-0d04-47d7-8b5c-d49eb2724e81 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:04.896897 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-32ad3998-ecec-4a54-b64b-a7771a7881ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:04.906626 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-32ad3998-ecec-4a54-b64b-a7771a7881ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1274 microversion: 2.1 time: 0.259788 Aug 07 01:12:04.907496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 97/193] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:04 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1274 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:05.306563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-81a265d1-0d04-47d7-8b5c-d49eb2724e81 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:05.337386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-81a265d1-0d04-47d7-8b5c-d49eb2724e81 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1499 microversion: 2.1 time: 0.487079 Aug 07 01:12:05.348794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 97/194] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:04 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1499 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:05.878302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2629dab1-83d9-4945-8531-ecf8d06ecfb9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:05.884124 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2629dab1-83d9-4945-8531-ecf8d06ecfb9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:05.913970 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2629dab1-83d9-4945-8531-ecf8d06ecfb9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:05.915833 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2629dab1-83d9-4945-8531-ecf8d06ecfb9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:05.919016 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2629dab1-83d9-4945-8531-ecf8d06ecfb9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:05.921494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2629dab1-83d9-4945-8531-ecf8d06ecfb9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:05.922552 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2629dab1-83d9-4945-8531-ecf8d06ecfb9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:05.923975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2629dab1-83d9-4945-8531-ecf8d06ecfb9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:05.942020 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-249ca8cb-6e39-4666-8eaf-a054342765f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:05.948298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-249ca8cb-6e39-4666-8eaf-a054342765f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:05.968423 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-249ca8cb-6e39-4666-8eaf-a054342765f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:05.969804 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-249ca8cb-6e39-4666-8eaf-a054342765f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:05.970612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-249ca8cb-6e39-4666-8eaf-a054342765f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:05.972452 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-249ca8cb-6e39-4666-8eaf-a054342765f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:05.974525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-249ca8cb-6e39-4666-8eaf-a054342765f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:05.976438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-249ca8cb-6e39-4666-8eaf-a054342765f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:06.327733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2629dab1-83d9-4945-8531-ecf8d06ecfb9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:06.367167 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2629dab1-83d9-4945-8531-ecf8d06ecfb9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:06.378042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2629dab1-83d9-4945-8531-ecf8d06ecfb9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:06.386605 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2629dab1-83d9-4945-8531-ecf8d06ecfb9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.513804 Aug 07 01:12:06.389302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 98/195] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:05 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:06.589264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-249ca8cb-6e39-4666-8eaf-a054342765f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:06.610673 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-249ca8cb-6e39-4666-8eaf-a054342765f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1280 microversion: 2.1 time: 0.679398 Aug 07 01:12:06.611756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 98/196] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:05 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1280 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:06.899532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7e396479-267f-4738-86fe-c728490f04f5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:06.900391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7e396479-267f-4738-86fe-c728490f04f5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:06.931590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7e396479-267f-4738-86fe-c728490f04f5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:06.933589 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7e396479-267f-4738-86fe-c728490f04f5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:06.936183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7e396479-267f-4738-86fe-c728490f04f5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:06.938543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7e396479-267f-4738-86fe-c728490f04f5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:06.940358 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7e396479-267f-4738-86fe-c728490f04f5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:06.940883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7e396479-267f-4738-86fe-c728490f04f5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:07.059790 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7e396479-267f-4738-86fe-c728490f04f5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Going to try to terminate instance {{(pid=117984) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:12:07.241454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=5,nova_cell1:INSERT=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:07.384047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-e0027765-daee-4778-a78e-0cabf3e285f5 req-ef247b1b-8dd2-4632-914d-5cce997b71ce service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "99e925dd-a3a5-44ab-bf20-9c80807d5530", "tag": "c91dc146-cf64-40b0-a3e9-644405de258c"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:07.435370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-e0027765-daee-4778-a78e-0cabf3e285f5 req-ef247b1b-8dd2-4632-914d-5cce997b71ce service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:07.437108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-e0027765-daee-4778-a78e-0cabf3e285f5 req-ef247b1b-8dd2-4632-914d-5cce997b71ce service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:07.438576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-e0027765-daee-4778-a78e-0cabf3e285f5 req-ef247b1b-8dd2-4632-914d-5cce997b71ce service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:07.447363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7e396479-267f-4738-86fe-c728490f04f5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "DELETE /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 204 len: 0 microversion: 2.1 time: 0.552003 Aug 07 01:12:07.459620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 99/197] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:12:06 2022] DELETE /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 07 01:12:07.470509 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-e0027765-daee-4778-a78e-0cabf3e285f5 req-ef247b1b-8dd2-4632-914d-5cce997b71ce service nova] Creating event network-changed:c91dc146-cf64-40b0-a3e9-644405de258c for instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 on hv-cinder-maste Aug 07 01:12:07.474393 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-baa1fc65-b2ad-46d2-92f9-633b2ee1b51f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:07.475613 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-baa1fc65-b2ad-46d2-92f9-633b2ee1b51f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:07.490122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-e0027765-daee-4778-a78e-0cabf3e285f5 req-ef247b1b-8dd2-4632-914d-5cce997b71ce service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.113662 Aug 07 01:12:07.491328 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 99/198] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:12:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:12:07.498511 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-baa1fc65-b2ad-46d2-92f9-633b2ee1b51f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:07.498751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-eada60fc-0bea-4a2d-b21c-68b7e37c4727 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:07.499677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-eada60fc-0bea-4a2d-b21c-68b7e37c4727 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:07.501742 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-baa1fc65-b2ad-46d2-92f9-633b2ee1b51f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:07.502228 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-baa1fc65-b2ad-46d2-92f9-633b2ee1b51f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:07.503668 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-baa1fc65-b2ad-46d2-92f9-633b2ee1b51f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:07.504251 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-baa1fc65-b2ad-46d2-92f9-633b2ee1b51f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:07.504934 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-baa1fc65-b2ad-46d2-92f9-633b2ee1b51f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:07.522221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eada60fc-0bea-4a2d-b21c-68b7e37c4727 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:07.523356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eada60fc-0bea-4a2d-b21c-68b7e37c4727 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:07.523846 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eada60fc-0bea-4a2d-b21c-68b7e37c4727 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:07.524711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eada60fc-0bea-4a2d-b21c-68b7e37c4727 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:07.525629 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eada60fc-0bea-4a2d-b21c-68b7e37c4727 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:07.526248 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eada60fc-0bea-4a2d-b21c-68b7e37c4727 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:07.936477 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-baa1fc65-b2ad-46d2-92f9-633b2ee1b51f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:07.947494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-baa1fc65-b2ad-46d2-92f9-633b2ee1b51f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:07.952421 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-baa1fc65-b2ad-46d2-92f9-633b2ee1b51f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:07.953652 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-baa1fc65-b2ad-46d2-92f9-633b2ee1b51f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.491329 Aug 07 01:12:07.957299 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 100/199] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:07 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:07.963144 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fb7c66f2-2608-4602-bfcf-f792bf1e02db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:07.963856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fb7c66f2-2608-4602-bfcf-f792bf1e02db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:07.985548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb7c66f2-2608-4602-bfcf-f792bf1e02db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:07.986165 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb7c66f2-2608-4602-bfcf-f792bf1e02db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:07.986488 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb7c66f2-2608-4602-bfcf-f792bf1e02db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:07.987484 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb7c66f2-2608-4602-bfcf-f792bf1e02db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:07.987863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb7c66f2-2608-4602-bfcf-f792bf1e02db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:07.988328 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb7c66f2-2608-4602-bfcf-f792bf1e02db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:08.109478 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:08.164191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-eada60fc-0bea-4a2d-b21c-68b7e37c4727 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:08.186418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-eada60fc-0bea-4a2d-b21c-68b7e37c4727 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1505 microversion: 2.1 time: 0.693525 Aug 07 01:12:08.188345 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 100/200] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:07 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1505 bytes in 696 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:08.318186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fb7c66f2-2608-4602-bfcf-f792bf1e02db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:08.338460 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fb7c66f2-2608-4602-bfcf-f792bf1e02db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.378845 Aug 07 01:12:08.339983 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 101/201] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:07 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:08.993831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-df00abaa-564e-4d27-b4dc-ffd82849bc11 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:08.994518 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-df00abaa-564e-4d27-b4dc-ffd82849bc11 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:09.017880 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df00abaa-564e-4d27-b4dc-ffd82849bc11 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:09.019030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df00abaa-564e-4d27-b4dc-ffd82849bc11 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:09.019486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df00abaa-564e-4d27-b4dc-ffd82849bc11 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:09.020221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df00abaa-564e-4d27-b4dc-ffd82849bc11 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:09.020708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df00abaa-564e-4d27-b4dc-ffd82849bc11 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:09.021137 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df00abaa-564e-4d27-b4dc-ffd82849bc11 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:09.231221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8788d8da-9c0a-4ede-ad1c-571f0c683806 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:09.232722 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8788d8da-9c0a-4ede-ad1c-571f0c683806 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:09.249790 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8788d8da-9c0a-4ede-ad1c-571f0c683806 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:09.250852 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8788d8da-9c0a-4ede-ad1c-571f0c683806 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:09.252063 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8788d8da-9c0a-4ede-ad1c-571f0c683806 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:09.253637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8788d8da-9c0a-4ede-ad1c-571f0c683806 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:09.254966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8788d8da-9c0a-4ede-ad1c-571f0c683806 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:09.255743 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8788d8da-9c0a-4ede-ad1c-571f0c683806 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:09.575802 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-df00abaa-564e-4d27-b4dc-ffd82849bc11 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:09.605678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-df00abaa-564e-4d27-b4dc-ffd82849bc11 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:09.611100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-df00abaa-564e-4d27-b4dc-ffd82849bc11 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:09.621011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-df00abaa-564e-4d27-b4dc-ffd82849bc11 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.626892 Aug 07 01:12:09.624134 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 101/202] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:08 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:09.626135 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8788d8da-9c0a-4ede-ad1c-571f0c683806 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:09.635543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e37dc633-49ac-4f8d-8ed1-16cf9a8d6e07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:09.638192 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e37dc633-49ac-4f8d-8ed1-16cf9a8d6e07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:09.648552 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8788d8da-9c0a-4ede-ad1c-571f0c683806 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1505 microversion: 2.1 time: 0.436869 Aug 07 01:12:09.650113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 102/203] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:09 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1505 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:09.661181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e37dc633-49ac-4f8d-8ed1-16cf9a8d6e07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:09.661181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e37dc633-49ac-4f8d-8ed1-16cf9a8d6e07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:09.661181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e37dc633-49ac-4f8d-8ed1-16cf9a8d6e07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:09.661181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e37dc633-49ac-4f8d-8ed1-16cf9a8d6e07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:09.661563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e37dc633-49ac-4f8d-8ed1-16cf9a8d6e07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:09.667095 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e37dc633-49ac-4f8d-8ed1-16cf9a8d6e07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:09.844329 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1355823236", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "04909372-4f17-4c44-b384-6e7c31434dac"}], "security_groups": [{"name": "tempest-securitygroup--2071856285"}], "key_name": "tempest-keypair-2078632662", "user_data": "}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:09.987470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e37dc633-49ac-4f8d-8ed1-16cf9a8d6e07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:10.009504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e37dc633-49ac-4f8d-8ed1-16cf9a8d6e07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.380042 Aug 07 01:12:10.022698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 102/204] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:09 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:10.095110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.network.neutron [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] validate_networks() for [('04909372-4f17-4c44-b384-6e7c31434dac', None, None, None, None, None)] {{(pid=117984) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:12:10.195247 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-f5f89d9b-fb46-4166-ae68-b68247d0ce8c req-8c9304e1-a1ae-4b7d-899f-f0efb10fe8ec service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3b71d249-75ce-4352-bb93-34e88cc0df93", "tag": "9ea29730-37e9-45db-98d4-1ef7d39b89d9"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:10.219119 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-f5f89d9b-fb46-4166-ae68-b68247d0ce8c req-8c9304e1-a1ae-4b7d-899f-f0efb10fe8ec service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:10.220067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-f5f89d9b-fb46-4166-ae68-b68247d0ce8c req-8c9304e1-a1ae-4b7d-899f-f0efb10fe8ec service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:10.220844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-f5f89d9b-fb46-4166-ae68-b68247d0ce8c req-8c9304e1-a1ae-4b7d-899f-f0efb10fe8ec service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:10.252805 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-f5f89d9b-fb46-4166-ae68-b68247d0ce8c req-8c9304e1-a1ae-4b7d-899f-f0efb10fe8ec service nova] Creating event network-changed:9ea29730-37e9-45db-98d4-1ef7d39b89d9 for instance 3b71d249-75ce-4352-bb93-34e88cc0df93 on hv-cinder-maste Aug 07 01:12:10.263441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-f5f89d9b-fb46-4166-ae68-b68247d0ce8c req-8c9304e1-a1ae-4b7d-899f-f0efb10fe8ec service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.076613 Aug 07 01:12:10.265465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 103/205] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:12:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:12:10.655456 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-070a3368-ea91-45d8-84c3-1218d6f02e32 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:10.656507 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-070a3368-ea91-45d8-84c3-1218d6f02e32 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:10.664044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-070a3368-ea91-45d8-84c3-1218d6f02e32 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:10.664588 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-070a3368-ea91-45d8-84c3-1218d6f02e32 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:10.665061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-070a3368-ea91-45d8-84c3-1218d6f02e32 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:10.665837 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-070a3368-ea91-45d8-84c3-1218d6f02e32 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:10.666408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-070a3368-ea91-45d8-84c3-1218d6f02e32 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:10.666969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-070a3368-ea91-45d8-84c3-1218d6f02e32 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:10.684855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:10.765061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=6 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:10.862310 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-070a3368-ea91-45d8-84c3-1218d6f02e32 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:10.882608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-070a3368-ea91-45d8-84c3-1218d6f02e32 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:10.884912 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-070a3368-ea91-45d8-84c3-1218d6f02e32 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:10.886325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-070a3368-ea91-45d8-84c3-1218d6f02e32 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.235986 Aug 07 01:12:10.887312 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 104/206] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:10 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:12:10.898461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-63eaa0e5-3ab8-492f-98c1-a42a76acd9f2 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:10.899828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-63eaa0e5-3ab8-492f-98c1-a42a76acd9f2 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:10.909750 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63eaa0e5-3ab8-492f-98c1-a42a76acd9f2 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:10.910563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63eaa0e5-3ab8-492f-98c1-a42a76acd9f2 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:10.911324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63eaa0e5-3ab8-492f-98c1-a42a76acd9f2 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:10.912925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63eaa0e5-3ab8-492f-98c1-a42a76acd9f2 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:10.914170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63eaa0e5-3ab8-492f-98c1-a42a76acd9f2 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:10.915127 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63eaa0e5-3ab8-492f-98c1-a42a76acd9f2 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:11.059550 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Flavor limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:12:11.060433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Image limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:12:11.060913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Flavor pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:12:11.061638 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Image pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:12:11.062318 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:12:11.084995 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:11.086925 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:11.087873 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:11.090370 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:11.091625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:11.092432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:11.122799 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for project 35820b4f79774ac49f43fce915f56d6f. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:12:11.128669 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for user e65f0f1ba54f483abff788974b1256cb and project 35820b4f79774ac49f43fce915f56d6f. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:12:11.148668 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Going to run 1 instances... {{(pid=117984) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:12:11.149638 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-63eaa0e5-3ab8-492f-98c1-a42a76acd9f2 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:11.151223 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117984) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:12:11.159250 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-63eaa0e5-3ab8-492f-98c1-a42a76acd9f2 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1505 microversion: 2.1 time: 0.270382 Aug 07 01:12:11.159825 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 105/207] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:10 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1505 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:11.180567 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a20c3097-8b21-4574-8a3d-f3bfeea73455 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:11.181492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a20c3097-8b21-4574-8a3d-f3bfeea73455 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:11.188938 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a20c3097-8b21-4574-8a3d-f3bfeea73455 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:11.189779 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a20c3097-8b21-4574-8a3d-f3bfeea73455 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:11.190991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a20c3097-8b21-4574-8a3d-f3bfeea73455 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:11.192141 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a20c3097-8b21-4574-8a3d-f3bfeea73455 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:11.192871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a20c3097-8b21-4574-8a3d-f3bfeea73455 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:11.193747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a20c3097-8b21-4574-8a3d-f3bfeea73455 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:11.230691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e0edfbda-5ff1-4589-9924-a9d3bd31e43c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.395677 Aug 07 01:12:11.231936 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 103/208] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:12:09 2022] POST /compute/v2.1/servers => generated 402 bytes in 1398 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:12:11.268711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e8e2fc30-3d20-4b46-aa50-340243d55787 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:11.272192 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e8e2fc30-3d20-4b46-aa50-340243d55787 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:11.453383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e8e2fc30-3d20-4b46-aa50-340243d55787 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:11.466691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e8e2fc30-3d20-4b46-aa50-340243d55787 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1242 microversion: 2.1 time: 0.223355 Aug 07 01:12:11.467144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a20c3097-8b21-4574-8a3d-f3bfeea73455 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:11.468710 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 104/209] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:11 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1242 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:11.486054 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a20c3097-8b21-4574-8a3d-f3bfeea73455 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.311310 Aug 07 01:12:11.487891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 106/210] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:11 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:11.928426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-62d80bf4-9aa1-4c4c-b248-90ee6f374246 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:11.929110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-62d80bf4-9aa1-4c4c-b248-90ee6f374246 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:11.936598 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62d80bf4-9aa1-4c4c-b248-90ee6f374246 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:11.937090 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62d80bf4-9aa1-4c4c-b248-90ee6f374246 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:11.937568 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62d80bf4-9aa1-4c4c-b248-90ee6f374246 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:11.938429 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62d80bf4-9aa1-4c4c-b248-90ee6f374246 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:11.939491 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62d80bf4-9aa1-4c4c-b248-90ee6f374246 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:11.940835 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62d80bf4-9aa1-4c4c-b248-90ee6f374246 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:12.141889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-62d80bf4-9aa1-4c4c-b248-90ee6f374246 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:12.152796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-62d80bf4-9aa1-4c4c-b248-90ee6f374246 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:12.154455 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-62d80bf4-9aa1-4c4c-b248-90ee6f374246 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:12.155852 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-62d80bf4-9aa1-4c4c-b248-90ee6f374246 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.233213 Aug 07 01:12:12.156568 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 105/211] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:11 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:12:12.186500 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e50bafa2-1efc-4444-b5d5-6e56a7bd1a4f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:12.187359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e50bafa2-1efc-4444-b5d5-6e56a7bd1a4f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:12.194814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e50bafa2-1efc-4444-b5d5-6e56a7bd1a4f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:12.195284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e50bafa2-1efc-4444-b5d5-6e56a7bd1a4f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:12.196008 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e50bafa2-1efc-4444-b5d5-6e56a7bd1a4f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:12.196602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e50bafa2-1efc-4444-b5d5-6e56a7bd1a4f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:12.197054 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e50bafa2-1efc-4444-b5d5-6e56a7bd1a4f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:12.197479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e50bafa2-1efc-4444-b5d5-6e56a7bd1a4f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:12.372596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e50bafa2-1efc-4444-b5d5-6e56a7bd1a4f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:12.381521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e50bafa2-1efc-4444-b5d5-6e56a7bd1a4f tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1505 microversion: 2.1 time: 0.198843 Aug 07 01:12:12.382294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 107/212] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:12 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1505 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:12.492349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-238d81cd-5442-43a2-a7ea-6b1399f4cb86 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:12.493277 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-238d81cd-5442-43a2-a7ea-6b1399f4cb86 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:12.500987 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-238d81cd-5442-43a2-a7ea-6b1399f4cb86 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:12.501612 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-238d81cd-5442-43a2-a7ea-6b1399f4cb86 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:12.502122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-238d81cd-5442-43a2-a7ea-6b1399f4cb86 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:12.503720 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-238d81cd-5442-43a2-a7ea-6b1399f4cb86 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:12.504552 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-238d81cd-5442-43a2-a7ea-6b1399f4cb86 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:12.505384 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-238d81cd-5442-43a2-a7ea-6b1399f4cb86 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:12.515146 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fdfa81fd-783d-4062-87f2-aeef7832aef2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:12.516696 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fdfa81fd-783d-4062-87f2-aeef7832aef2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:12.529200 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fdfa81fd-783d-4062-87f2-aeef7832aef2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:12.530376 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fdfa81fd-783d-4062-87f2-aeef7832aef2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:12.531408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fdfa81fd-783d-4062-87f2-aeef7832aef2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:12.532964 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fdfa81fd-783d-4062-87f2-aeef7832aef2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:12.534035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fdfa81fd-783d-4062-87f2-aeef7832aef2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:12.535063 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fdfa81fd-783d-4062-87f2-aeef7832aef2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:12.544030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:12.610160 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-238d81cd-5442-43a2-a7ea-6b1399f4cb86 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:12.623218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-238d81cd-5442-43a2-a7ea-6b1399f4cb86 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1246 microversion: 2.1 time: 0.134710 Aug 07 01:12:12.623976 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 106/213] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:12 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1246 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:12.629515 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1055904786", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452"}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:12.703069 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.network.neutron [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] validate_networks() for None {{(pid=117984) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:12:12.771819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-fdfa81fd-783d-4062-87f2-aeef7832aef2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:12.779609 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fdfa81fd-783d-4062-87f2-aeef7832aef2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.272098 Aug 07 01:12:12.780204 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 108/214] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:12 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:13.130286 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Flavor limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:12:13.131288 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Image limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:12:13.132206 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Flavor pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:12:13.133134 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Image pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:12:13.133703 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:12:13.149669 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:13.150554 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:13.151501 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:13.152852 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:13.153633 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:13.154811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:13.186039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f9e2ddf5-4d90-48f8-b092-e462ae6709d4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:13.186473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Getting quotas for project 4203fc81bbd14439bb6ce329a5ccbc79. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:12:13.187905 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f9e2ddf5-4d90-48f8-b092-e462ae6709d4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:13.192029 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Getting quotas for user 0ab3ed0f7df742618c77815ef1ddb960 and project 4203fc81bbd14439bb6ce329a5ccbc79. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:12:13.196030 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=7,nova_api:UPDATE=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:13.200973 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9e2ddf5-4d90-48f8-b092-e462ae6709d4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:13.202262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9e2ddf5-4d90-48f8-b092-e462ae6709d4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:13.203252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9e2ddf5-4d90-48f8-b092-e462ae6709d4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:13.204703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9e2ddf5-4d90-48f8-b092-e462ae6709d4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:13.206722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9e2ddf5-4d90-48f8-b092-e462ae6709d4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:13.207788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9e2ddf5-4d90-48f8-b092-e462ae6709d4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:13.213336 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Going to run 1 instances... {{(pid=117984) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:12:13.216956 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117984) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:12:13.296513 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-07847ff9-700c-40e3-a716-89bc0181ad33 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.670742 Aug 07 01:12:13.297740 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 107/215] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:12:12 2022] POST /compute/v2.1/servers => generated 380 bytes in 673 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:12:13.326714 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8ccc1a6e-21f7-488d-80f4-c850d3f88b20 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:13.327979 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8ccc1a6e-21f7-488d-80f4-c850d3f88b20 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:13.481846 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8ccc1a6e-21f7-488d-80f4-c850d3f88b20 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:13.498201 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8ccc1a6e-21f7-488d-80f4-c850d3f88b20 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1215 microversion: 2.1 time: 0.185621 Aug 07 01:12:13.499655 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 108/216] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:13 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1215 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:13.516976 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d196b78f-aa45-45ed-887f-3562c52d0ccb tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:13.519913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d196b78f-aa45-45ed-887f-3562c52d0ccb tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:13.535137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d196b78f-aa45-45ed-887f-3562c52d0ccb tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:13.536766 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d196b78f-aa45-45ed-887f-3562c52d0ccb tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:13.537965 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d196b78f-aa45-45ed-887f-3562c52d0ccb tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:13.540106 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d196b78f-aa45-45ed-887f-3562c52d0ccb tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:13.541656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d196b78f-aa45-45ed-887f-3562c52d0ccb tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:13.542740 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d196b78f-aa45-45ed-887f-3562c52d0ccb tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:13.615766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f9e2ddf5-4d90-48f8-b092-e462ae6709d4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:13.637288 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f9e2ddf5-4d90-48f8-b092-e462ae6709d4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:13.638940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f9e2ddf5-4d90-48f8-b092-e462ae6709d4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:13.641361 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f9e2ddf5-4d90-48f8-b092-e462ae6709d4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.463460 Aug 07 01:12:13.645334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 109/217] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:13 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:13.653760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b4762c3a-a033-4d57-80fc-1bf7bbba5960 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:13.654907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b4762c3a-a033-4d57-80fc-1bf7bbba5960 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:13.674294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b4762c3a-a033-4d57-80fc-1bf7bbba5960 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:13.676463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b4762c3a-a033-4d57-80fc-1bf7bbba5960 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:13.677678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b4762c3a-a033-4d57-80fc-1bf7bbba5960 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:13.679355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b4762c3a-a033-4d57-80fc-1bf7bbba5960 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:13.680326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b4762c3a-a033-4d57-80fc-1bf7bbba5960 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:13.680988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b4762c3a-a033-4d57-80fc-1bf7bbba5960 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:13.814231 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b4762c3a-a033-4d57-80fc-1bf7bbba5960 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:13.830214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b4762c3a-a033-4d57-80fc-1bf7bbba5960 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1246 microversion: 2.1 time: 0.183291 Aug 07 01:12:13.830904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 110/218] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:13 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1246 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:13.841605 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4970c4c1-fb65-487d-9cbd-019c465b7044 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:13.842857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4970c4c1-fb65-487d-9cbd-019c465b7044 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:13.844049 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d196b78f-aa45-45ed-887f-3562c52d0ccb tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:13.854983 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4970c4c1-fb65-487d-9cbd-019c465b7044 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:13.855806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4970c4c1-fb65-487d-9cbd-019c465b7044 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:13.856754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4970c4c1-fb65-487d-9cbd-019c465b7044 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:13.856996 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d196b78f-aa45-45ed-887f-3562c52d0ccb tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1505 microversion: 2.1 time: 0.355568 Aug 07 01:12:13.857798 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4970c4c1-fb65-487d-9cbd-019c465b7044 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:13.857997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 109/219] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:13 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1505 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:13.858658 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4970c4c1-fb65-487d-9cbd-019c465b7044 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:13.859415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4970c4c1-fb65-487d-9cbd-019c465b7044 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:14.060945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4970c4c1-fb65-487d-9cbd-019c465b7044 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:14.069377 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4970c4c1-fb65-487d-9cbd-019c465b7044 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.235072 Aug 07 01:12:14.070112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 111/220] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:13 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:14.538114 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b17fb80a-5481-4958-828c-72c2d08ad53b tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:14.539480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b17fb80a-5481-4958-828c-72c2d08ad53b tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:14.555583 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b17fb80a-5481-4958-828c-72c2d08ad53b tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:14.556543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b17fb80a-5481-4958-828c-72c2d08ad53b tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:14.557564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b17fb80a-5481-4958-828c-72c2d08ad53b tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:14.558948 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b17fb80a-5481-4958-828c-72c2d08ad53b tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:14.559938 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b17fb80a-5481-4958-828c-72c2d08ad53b tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:14.561055 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b17fb80a-5481-4958-828c-72c2d08ad53b tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:14.662425 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b17fb80a-5481-4958-828c-72c2d08ad53b tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:14.672797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b17fb80a-5481-4958-828c-72c2d08ad53b tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1219 microversion: 2.1 time: 0.142342 Aug 07 01:12:14.673821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 110/221] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:14 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1219 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:14.698687 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-138fa769-6ff7-4c0b-aaf7-4a265b450382 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:14.699796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-138fa769-6ff7-4c0b-aaf7-4a265b450382 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:14.711215 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-138fa769-6ff7-4c0b-aaf7-4a265b450382 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:14.712049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-138fa769-6ff7-4c0b-aaf7-4a265b450382 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:14.712848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-138fa769-6ff7-4c0b-aaf7-4a265b450382 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:14.714008 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-138fa769-6ff7-4c0b-aaf7-4a265b450382 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:14.714761 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-138fa769-6ff7-4c0b-aaf7-4a265b450382 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:14.715458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-138fa769-6ff7-4c0b-aaf7-4a265b450382 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:14.856034 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-80a94634-b85d-44bd-a837-c3759f912bb1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:14.857133 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-80a94634-b85d-44bd-a837-c3759f912bb1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:14.868758 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80a94634-b85d-44bd-a837-c3759f912bb1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:14.870002 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80a94634-b85d-44bd-a837-c3759f912bb1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:14.870785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80a94634-b85d-44bd-a837-c3759f912bb1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:14.872451 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80a94634-b85d-44bd-a837-c3759f912bb1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:14.873244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80a94634-b85d-44bd-a837-c3759f912bb1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:14.874181 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80a94634-b85d-44bd-a837-c3759f912bb1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:14.958268 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-138fa769-6ff7-4c0b-aaf7-4a265b450382 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:14.973834 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-138fa769-6ff7-4c0b-aaf7-4a265b450382 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:14.977555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-138fa769-6ff7-4c0b-aaf7-4a265b450382 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:14.979965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-138fa769-6ff7-4c0b-aaf7-4a265b450382 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.291730 Aug 07 01:12:14.981128 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 112/222] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:14 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:14.998049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ee9fe4f3-44ea-4452-beef-cd0674ddbdd5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:14.998721 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ee9fe4f3-44ea-4452-beef-cd0674ddbdd5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:15.007013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee9fe4f3-44ea-4452-beef-cd0674ddbdd5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:15.007517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee9fe4f3-44ea-4452-beef-cd0674ddbdd5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:15.008505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee9fe4f3-44ea-4452-beef-cd0674ddbdd5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:15.009258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee9fe4f3-44ea-4452-beef-cd0674ddbdd5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:15.009998 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee9fe4f3-44ea-4452-beef-cd0674ddbdd5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:15.010786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee9fe4f3-44ea-4452-beef-cd0674ddbdd5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:15.021588 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-80a94634-b85d-44bd-a837-c3759f912bb1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:15.036563 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-80a94634-b85d-44bd-a837-c3759f912bb1 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1246 microversion: 2.1 time: 0.186643 Aug 07 01:12:15.037623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 111/223] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:14 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1246 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:15.098472 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2f7dc007-9e52-4def-bc65-15f5072faf27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:15.099591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2f7dc007-9e52-4def-bc65-15f5072faf27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:15.110398 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f7dc007-9e52-4def-bc65-15f5072faf27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:15.111227 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f7dc007-9e52-4def-bc65-15f5072faf27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:15.112617 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f7dc007-9e52-4def-bc65-15f5072faf27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:15.113789 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f7dc007-9e52-4def-bc65-15f5072faf27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:15.114394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f7dc007-9e52-4def-bc65-15f5072faf27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:15.114945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f7dc007-9e52-4def-bc65-15f5072faf27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:15.225167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ee9fe4f3-44ea-4452-beef-cd0674ddbdd5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:15.239196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ee9fe4f3-44ea-4452-beef-cd0674ddbdd5 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1505 microversion: 2.1 time: 0.245721 Aug 07 01:12:15.244712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 113/224] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:14 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1505 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:15.348690 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2f7dc007-9e52-4def-bc65-15f5072faf27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:15.368170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2f7dc007-9e52-4def-bc65-15f5072faf27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.276227 Aug 07 01:12:15.369499 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 112/225] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:15 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:15.706324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2b6b8292-3185-461d-abf4-9abbfcefedf9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:15.707701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2b6b8292-3185-461d-abf4-9abbfcefedf9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:15.719963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b6b8292-3185-461d-abf4-9abbfcefedf9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:15.720836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b6b8292-3185-461d-abf4-9abbfcefedf9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:15.721740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b6b8292-3185-461d-abf4-9abbfcefedf9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:15.723055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b6b8292-3185-461d-abf4-9abbfcefedf9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:15.723912 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b6b8292-3185-461d-abf4-9abbfcefedf9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:15.724738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b6b8292-3185-461d-abf4-9abbfcefedf9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:15.844690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2b6b8292-3185-461d-abf4-9abbfcefedf9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:15.859165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2b6b8292-3185-461d-abf4-9abbfcefedf9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1219 microversion: 2.1 time: 0.159901 Aug 07 01:12:15.860600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 114/226] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:15 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1219 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:16.033657 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-30586d8f-6d25-4d4b-9048-f49d5d8ed062 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:16.034731 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-30586d8f-6d25-4d4b-9048-f49d5d8ed062 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:16.046152 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-30586d8f-6d25-4d4b-9048-f49d5d8ed062 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:16.047027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-30586d8f-6d25-4d4b-9048-f49d5d8ed062 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:16.047809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-30586d8f-6d25-4d4b-9048-f49d5d8ed062 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:16.049698 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-30586d8f-6d25-4d4b-9048-f49d5d8ed062 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:16.056446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-30586d8f-6d25-4d4b-9048-f49d5d8ed062 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:16.056446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-30586d8f-6d25-4d4b-9048-f49d5d8ed062 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:16.075949 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-168a1feb-11af-4487-a947-cae2a802f685 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:16.076759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-168a1feb-11af-4487-a947-cae2a802f685 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:16.084723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-168a1feb-11af-4487-a947-cae2a802f685 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:16.085230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-168a1feb-11af-4487-a947-cae2a802f685 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:16.085712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-168a1feb-11af-4487-a947-cae2a802f685 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:16.086499 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-168a1feb-11af-4487-a947-cae2a802f685 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:16.087083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-168a1feb-11af-4487-a947-cae2a802f685 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:16.087631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-168a1feb-11af-4487-a947-cae2a802f685 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:16.186774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-168a1feb-11af-4487-a947-cae2a802f685 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:16.202408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-168a1feb-11af-4487-a947-cae2a802f685 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1246 microversion: 2.1 time: 0.132967 Aug 07 01:12:16.203633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 115/227] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:16 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1246 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:16.276988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-94930db9-e4c3-4b74-ae13-ee23b6a1098b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:16.278169 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-94930db9-e4c3-4b74-ae13-ee23b6a1098b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:16.287049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94930db9-e4c3-4b74-ae13-ee23b6a1098b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:16.287957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94930db9-e4c3-4b74-ae13-ee23b6a1098b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:16.288466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94930db9-e4c3-4b74-ae13-ee23b6a1098b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:16.289238 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94930db9-e4c3-4b74-ae13-ee23b6a1098b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:16.289701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94930db9-e4c3-4b74-ae13-ee23b6a1098b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:16.290337 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94930db9-e4c3-4b74-ae13-ee23b6a1098b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:16.474851 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-30586d8f-6d25-4d4b-9048-f49d5d8ed062 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:16.486284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-94930db9-e4c3-4b74-ae13-ee23b6a1098b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:16.493823 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-30586d8f-6d25-4d4b-9048-f49d5d8ed062 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:16.495751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-94930db9-e4c3-4b74-ae13-ee23b6a1098b tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1505 microversion: 2.1 time: 0.223497 Aug 07 01:12:16.497113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 116/228] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:16 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1505 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:16.497492 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-30586d8f-6d25-4d4b-9048-f49d5d8ed062 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:16.499982 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-30586d8f-6d25-4d4b-9048-f49d5d8ed062 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.473008 Aug 07 01:12:16.501469 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 113/229] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:16 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:16.514267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-09de156d-171b-44f0-b6d0-32e2dc18fe9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:16.515422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-09de156d-171b-44f0-b6d0-32e2dc18fe9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:16.527940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09de156d-171b-44f0-b6d0-32e2dc18fe9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:16.529212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09de156d-171b-44f0-b6d0-32e2dc18fe9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:16.530260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09de156d-171b-44f0-b6d0-32e2dc18fe9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:16.531597 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09de156d-171b-44f0-b6d0-32e2dc18fe9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:16.532487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09de156d-171b-44f0-b6d0-32e2dc18fe9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:16.533090 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09de156d-171b-44f0-b6d0-32e2dc18fe9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:16.767101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-09de156d-171b-44f0-b6d0-32e2dc18fe9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:16.783147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-09de156d-171b-44f0-b6d0-32e2dc18fe9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.276298 Aug 07 01:12:16.785269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 117/230] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:16 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:16.908730 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5aed5970-090c-409e-ac38-5b1ef37c41d0 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:16.910309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5aed5970-090c-409e-ac38-5b1ef37c41d0 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:16.918472 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5aed5970-090c-409e-ac38-5b1ef37c41d0 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:16.919215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5aed5970-090c-409e-ac38-5b1ef37c41d0 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:16.919883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5aed5970-090c-409e-ac38-5b1ef37c41d0 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:16.920780 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5aed5970-090c-409e-ac38-5b1ef37c41d0 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:16.921341 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5aed5970-090c-409e-ac38-5b1ef37c41d0 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:16.921880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5aed5970-090c-409e-ac38-5b1ef37c41d0 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:17.009537 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5aed5970-090c-409e-ac38-5b1ef37c41d0 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:17.021629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5aed5970-090c-409e-ac38-5b1ef37c41d0 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1219 microversion: 2.1 time: 0.122693 Aug 07 01:12:17.022275 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 114/231] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:16 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1219 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:17.234118 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3d289562-c9ed-473f-b890-006fede5a650 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:17.235390 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3d289562-c9ed-473f-b890-006fede5a650 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:17.247851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d289562-c9ed-473f-b890-006fede5a650 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:17.248720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d289562-c9ed-473f-b890-006fede5a650 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:17.249479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d289562-c9ed-473f-b890-006fede5a650 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:17.250675 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d289562-c9ed-473f-b890-006fede5a650 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:17.251482 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d289562-c9ed-473f-b890-006fede5a650 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:17.252193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d289562-c9ed-473f-b890-006fede5a650 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:17.360825 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3d289562-c9ed-473f-b890-006fede5a650 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:17.375189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3d289562-c9ed-473f-b890-006fede5a650 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1246 microversion: 2.1 time: 0.149233 Aug 07 01:12:17.383889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 118/232] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:17 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1246 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:17.518557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1204c88b-b79b-4b03-877d-5fb95052d6cd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:17.519381 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1204c88b-b79b-4b03-877d-5fb95052d6cd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:17.526215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1204c88b-b79b-4b03-877d-5fb95052d6cd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:17.526793 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1204c88b-b79b-4b03-877d-5fb95052d6cd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:17.527911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1204c88b-b79b-4b03-877d-5fb95052d6cd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:17.528065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1204c88b-b79b-4b03-877d-5fb95052d6cd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:17.528764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1204c88b-b79b-4b03-877d-5fb95052d6cd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:17.529217 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1204c88b-b79b-4b03-877d-5fb95052d6cd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:17.539692 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-347d4e68-2b73-4dea-aa48-76b470d631b7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:17.540324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-347d4e68-2b73-4dea-aa48-76b470d631b7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:17.542268 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=83,nova_api:INSERT=3 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:17.548178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-347d4e68-2b73-4dea-aa48-76b470d631b7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:17.548746 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-347d4e68-2b73-4dea-aa48-76b470d631b7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:17.549134 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-347d4e68-2b73-4dea-aa48-76b470d631b7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:17.549900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-347d4e68-2b73-4dea-aa48-76b470d631b7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:17.550357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-347d4e68-2b73-4dea-aa48-76b470d631b7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:17.550784 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-347d4e68-2b73-4dea-aa48-76b470d631b7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:17.726909 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1204c88b-b79b-4b03-877d-5fb95052d6cd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:17.746864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-347d4e68-2b73-4dea-aa48-76b470d631b7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:17.748410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1204c88b-b79b-4b03-877d-5fb95052d6cd tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1505 microversion: 2.1 time: 0.234131 Aug 07 01:12:17.749435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 115/233] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:17 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1505 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:17.756615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-347d4e68-2b73-4dea-aa48-76b470d631b7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:17.762838 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-347d4e68-2b73-4dea-aa48-76b470d631b7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:17.767560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-347d4e68-2b73-4dea-aa48-76b470d631b7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.231165 Aug 07 01:12:17.769250 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 119/234] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:17 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:12:17.821109 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a1879e3e-f9e0-4ab4-8ec9-d93bc33c2958 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:17.822645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a1879e3e-f9e0-4ab4-8ec9-d93bc33c2958 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:17.834781 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a1879e3e-f9e0-4ab4-8ec9-d93bc33c2958 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:17.836220 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a1879e3e-f9e0-4ab4-8ec9-d93bc33c2958 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:17.837914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a1879e3e-f9e0-4ab4-8ec9-d93bc33c2958 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:17.840197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a1879e3e-f9e0-4ab4-8ec9-d93bc33c2958 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:17.841606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a1879e3e-f9e0-4ab4-8ec9-d93bc33c2958 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:17.842541 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a1879e3e-f9e0-4ab4-8ec9-d93bc33c2958 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:17.923911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-297dc6eb-b5b8-44dc-8608-5955474c01ef req-557f5473-e818-4ea7-a32b-bbf0846ca605 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "95d224a0-7a33-422a-bae6-06f4acfac258", "name": "network-vif-unplugged", "status": "completed", "tag": "0108d266-48bb-4aec-97ba-3cbd0446ef81"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:17.942710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-297dc6eb-b5b8-44dc-8608-5955474c01ef req-557f5473-e818-4ea7-a32b-bbf0846ca605 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:17.944013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-297dc6eb-b5b8-44dc-8608-5955474c01ef req-557f5473-e818-4ea7-a32b-bbf0846ca605 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:17.944174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-297dc6eb-b5b8-44dc-8608-5955474c01ef req-557f5473-e818-4ea7-a32b-bbf0846ca605 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:17.965415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-297dc6eb-b5b8-44dc-8608-5955474c01ef req-557f5473-e818-4ea7-a32b-bbf0846ca605 service nova] Creating event network-vif-unplugged:0108d266-48bb-4aec-97ba-3cbd0446ef81 for instance 95d224a0-7a33-422a-bae6-06f4acfac258 on hv-cinder-maste Aug 07 01:12:17.973030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-297dc6eb-b5b8-44dc-8608-5955474c01ef req-557f5473-e818-4ea7-a32b-bbf0846ca605 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.054819 Aug 07 01:12:17.974548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 120/235] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:12:17 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:12:18.053469 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8840bcba-227e-4a1d-9891-1f7937c8d6ee tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:18.055023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8840bcba-227e-4a1d-9891-1f7937c8d6ee tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:18.093422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8840bcba-227e-4a1d-9891-1f7937c8d6ee tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:18.094418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8840bcba-227e-4a1d-9891-1f7937c8d6ee tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:18.095330 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8840bcba-227e-4a1d-9891-1f7937c8d6ee tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:18.097888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8840bcba-227e-4a1d-9891-1f7937c8d6ee tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:18.098378 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8840bcba-227e-4a1d-9891-1f7937c8d6ee tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:18.099474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8840bcba-227e-4a1d-9891-1f7937c8d6ee tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:18.156718 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a1879e3e-f9e0-4ab4-8ec9-d93bc33c2958 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:18.170516 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a1879e3e-f9e0-4ab4-8ec9-d93bc33c2958 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.356127 Aug 07 01:12:18.171313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 116/236] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:17 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:18.241839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8840bcba-227e-4a1d-9891-1f7937c8d6ee tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:18.259644 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8840bcba-227e-4a1d-9891-1f7937c8d6ee tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1219 microversion: 2.1 time: 0.215879 Aug 07 01:12:18.262014 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 121/237] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:18 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1219 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:18.403816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bdab0fd0-e243-4c3c-99f2-eb97d2cde563 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:18.405062 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bdab0fd0-e243-4c3c-99f2-eb97d2cde563 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:18.415961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bdab0fd0-e243-4c3c-99f2-eb97d2cde563 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:18.416805 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bdab0fd0-e243-4c3c-99f2-eb97d2cde563 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:18.417529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bdab0fd0-e243-4c3c-99f2-eb97d2cde563 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:18.419029 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bdab0fd0-e243-4c3c-99f2-eb97d2cde563 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:18.419798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bdab0fd0-e243-4c3c-99f2-eb97d2cde563 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:18.420752 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bdab0fd0-e243-4c3c-99f2-eb97d2cde563 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:18.573082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bdab0fd0-e243-4c3c-99f2-eb97d2cde563 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:18.573082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bdab0fd0-e243-4c3c-99f2-eb97d2cde563 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1246 microversion: 2.1 time: 0.168395 Aug 07 01:12:18.573082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 117/238] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:18 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1246 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:18.787714 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2c464f44-c7ee-469a-8ca0-5c6acaf62dea tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:18.789095 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2c464f44-c7ee-469a-8ca0-5c6acaf62dea tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:18.794140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-668eaec6-4ba3-4795-af2c-642fdb7b1a73 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:18.795642 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-668eaec6-4ba3-4795-af2c-642fdb7b1a73 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:18.804874 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-668eaec6-4ba3-4795-af2c-642fdb7b1a73 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:18.805351 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-668eaec6-4ba3-4795-af2c-642fdb7b1a73 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:18.806082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-668eaec6-4ba3-4795-af2c-642fdb7b1a73 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:18.807179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-668eaec6-4ba3-4795-af2c-642fdb7b1a73 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:18.807912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-668eaec6-4ba3-4795-af2c-642fdb7b1a73 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:18.808559 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-668eaec6-4ba3-4795-af2c-642fdb7b1a73 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:18.809075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2c464f44-c7ee-469a-8ca0-5c6acaf62dea tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:18.810029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2c464f44-c7ee-469a-8ca0-5c6acaf62dea tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:18.811484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2c464f44-c7ee-469a-8ca0-5c6acaf62dea tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:18.812353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2c464f44-c7ee-469a-8ca0-5c6acaf62dea tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:18.813024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2c464f44-c7ee-469a-8ca0-5c6acaf62dea tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:18.813781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2c464f44-c7ee-469a-8ca0-5c6acaf62dea tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:19.055674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2c464f44-c7ee-469a-8ca0-5c6acaf62dea tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:19.071320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2c464f44-c7ee-469a-8ca0-5c6acaf62dea tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1505 microversion: 2.1 time: 0.287690 Aug 07 01:12:19.072744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 122/239] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:18 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1505 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:19.161138 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-668eaec6-4ba3-4795-af2c-642fdb7b1a73 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:19.173891 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-668eaec6-4ba3-4795-af2c-642fdb7b1a73 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:19.177739 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-668eaec6-4ba3-4795-af2c-642fdb7b1a73 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:19.179555 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-668eaec6-4ba3-4795-af2c-642fdb7b1a73 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.391207 Aug 07 01:12:19.180314 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 118/240] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:18 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:19.223087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-26fe1d6a-d555-4edb-bdbb-077657750c6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:19.224524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-26fe1d6a-d555-4edb-bdbb-077657750c6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:19.232646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26fe1d6a-d555-4edb-bdbb-077657750c6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:19.233289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26fe1d6a-d555-4edb-bdbb-077657750c6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:19.233907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26fe1d6a-d555-4edb-bdbb-077657750c6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:19.234727 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26fe1d6a-d555-4edb-bdbb-077657750c6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:19.235275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26fe1d6a-d555-4edb-bdbb-077657750c6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:19.235870 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26fe1d6a-d555-4edb-bdbb-077657750c6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:19.300098 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8fef1bbf-3a72-4238-b939-4fc8d1a9ee20 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:19.301467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8fef1bbf-3a72-4238-b939-4fc8d1a9ee20 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:19.313500 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8fef1bbf-3a72-4238-b939-4fc8d1a9ee20 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:19.314932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8fef1bbf-3a72-4238-b939-4fc8d1a9ee20 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:19.316371 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8fef1bbf-3a72-4238-b939-4fc8d1a9ee20 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:19.317854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8fef1bbf-3a72-4238-b939-4fc8d1a9ee20 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:19.319047 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8fef1bbf-3a72-4238-b939-4fc8d1a9ee20 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:19.319716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8fef1bbf-3a72-4238-b939-4fc8d1a9ee20 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:19.324293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:19.413332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8fef1bbf-3a72-4238-b939-4fc8d1a9ee20 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:19.425263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8fef1bbf-3a72-4238-b939-4fc8d1a9ee20 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1219 microversion: 2.1 time: 0.131802 Aug 07 01:12:19.436959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 119/241] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:19 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1219 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:19.472128 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-26fe1d6a-d555-4edb-bdbb-077657750c6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:19.483954 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-26fe1d6a-d555-4edb-bdbb-077657750c6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.268401 Aug 07 01:12:19.485054 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 123/242] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:19 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:19.594038 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fe6e6498-ab30-46a2-a5e5-5f930727b715 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:19.595183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fe6e6498-ab30-46a2-a5e5-5f930727b715 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:19.607049 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe6e6498-ab30-46a2-a5e5-5f930727b715 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:19.608334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe6e6498-ab30-46a2-a5e5-5f930727b715 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:19.609423 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe6e6498-ab30-46a2-a5e5-5f930727b715 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:19.610774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe6e6498-ab30-46a2-a5e5-5f930727b715 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:19.611656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe6e6498-ab30-46a2-a5e5-5f930727b715 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:19.612460 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe6e6498-ab30-46a2-a5e5-5f930727b715 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:19.705007 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fe6e6498-ab30-46a2-a5e5-5f930727b715 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:19.719966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fe6e6498-ab30-46a2-a5e5-5f930727b715 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1246 microversion: 2.1 time: 0.133308 Aug 07 01:12:19.722476 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 120/243] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:19 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1246 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:20.103141 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-06b53172-2c01-4287-bdbd-0809f40fb2c9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:20.106039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-06b53172-2c01-4287-bdbd-0809f40fb2c9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:20.119257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-06b53172-2c01-4287-bdbd-0809f40fb2c9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:20.120149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-06b53172-2c01-4287-bdbd-0809f40fb2c9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:20.120993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-06b53172-2c01-4287-bdbd-0809f40fb2c9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:20.122361 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-06b53172-2c01-4287-bdbd-0809f40fb2c9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:20.123287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-06b53172-2c01-4287-bdbd-0809f40fb2c9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:20.124043 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-06b53172-2c01-4287-bdbd-0809f40fb2c9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:20.213094 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e583d854-04f4-488c-9d1a-365933a1dc88 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:20.214417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e583d854-04f4-488c-9d1a-365933a1dc88 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:20.225955 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e583d854-04f4-488c-9d1a-365933a1dc88 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:20.226787 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e583d854-04f4-488c-9d1a-365933a1dc88 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:20.227580 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e583d854-04f4-488c-9d1a-365933a1dc88 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:20.229340 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e583d854-04f4-488c-9d1a-365933a1dc88 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:20.230386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e583d854-04f4-488c-9d1a-365933a1dc88 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:20.231675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e583d854-04f4-488c-9d1a-365933a1dc88 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:20.359559 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-06b53172-2c01-4287-bdbd-0809f40fb2c9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:20.374428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-06b53172-2c01-4287-bdbd-0809f40fb2c9 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1505 microversion: 2.1 time: 0.279121 Aug 07 01:12:20.375736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 124/244] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:20 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1505 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:20.446512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e583d854-04f4-488c-9d1a-365933a1dc88 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:20.453400 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4d7bb13b-b7cb-432d-ab29-f8212c63b372 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:20.454406 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4d7bb13b-b7cb-432d-ab29-f8212c63b372 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:20.465733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e583d854-04f4-488c-9d1a-365933a1dc88 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:20.466216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4d7bb13b-b7cb-432d-ab29-f8212c63b372 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:20.467291 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4d7bb13b-b7cb-432d-ab29-f8212c63b372 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:20.468516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4d7bb13b-b7cb-432d-ab29-f8212c63b372 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:20.469160 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e583d854-04f4-488c-9d1a-365933a1dc88 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:20.469866 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4d7bb13b-b7cb-432d-ab29-f8212c63b372 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:20.471535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4d7bb13b-b7cb-432d-ab29-f8212c63b372 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:20.472304 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e583d854-04f4-488c-9d1a-365933a1dc88 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.267515 Aug 07 01:12:20.472498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4d7bb13b-b7cb-432d-ab29-f8212c63b372 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:20.488803 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 121/245] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:20 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:20.503141 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:20.522400 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-39f02935-28cf-4726-932c-0162a6b3ebc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:20.523843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-39f02935-28cf-4726-932c-0162a6b3ebc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:20.539512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39f02935-28cf-4726-932c-0162a6b3ebc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:20.540729 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39f02935-28cf-4726-932c-0162a6b3ebc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:20.542456 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39f02935-28cf-4726-932c-0162a6b3ebc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:20.544478 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39f02935-28cf-4726-932c-0162a6b3ebc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:20.545620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39f02935-28cf-4726-932c-0162a6b3ebc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:20.546651 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39f02935-28cf-4726-932c-0162a6b3ebc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:20.591290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4d7bb13b-b7cb-432d-ab29-f8212c63b372 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:20.604453 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4d7bb13b-b7cb-432d-ab29-f8212c63b372 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1219 microversion: 2.1 time: 0.157717 Aug 07 01:12:20.605749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 125/246] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:20 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1219 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:20.752002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5c45ec21-3f6c-477e-93ff-0333b1e5175c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:20.753739 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5c45ec21-3f6c-477e-93ff-0333b1e5175c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:20.765842 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5c45ec21-3f6c-477e-93ff-0333b1e5175c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:20.767135 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5c45ec21-3f6c-477e-93ff-0333b1e5175c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:20.768502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5c45ec21-3f6c-477e-93ff-0333b1e5175c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:20.770272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5c45ec21-3f6c-477e-93ff-0333b1e5175c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:20.771083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5c45ec21-3f6c-477e-93ff-0333b1e5175c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:20.771777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5c45ec21-3f6c-477e-93ff-0333b1e5175c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:20.791648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-39f02935-28cf-4726-932c-0162a6b3ebc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:20.813217 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-39f02935-28cf-4726-932c-0162a6b3ebc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.307563 Aug 07 01:12:20.814994 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 122/247] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:20 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:20.893839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5c45ec21-3f6c-477e-93ff-0333b1e5175c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:20.908907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5c45ec21-3f6c-477e-93ff-0333b1e5175c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1246 microversion: 2.1 time: 0.164911 Aug 07 01:12:20.918480 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 126/248] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:20 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1246 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:21.398611 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6c32a815-08d4-4ff4-b7aa-eb52e1c619de tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:21.399278 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6c32a815-08d4-4ff4-b7aa-eb52e1c619de tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:21.406454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c32a815-08d4-4ff4-b7aa-eb52e1c619de tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:21.407003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c32a815-08d4-4ff4-b7aa-eb52e1c619de tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:21.407483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c32a815-08d4-4ff4-b7aa-eb52e1c619de tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:21.408400 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c32a815-08d4-4ff4-b7aa-eb52e1c619de tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:21.409032 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c32a815-08d4-4ff4-b7aa-eb52e1c619de tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:21.409509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c32a815-08d4-4ff4-b7aa-eb52e1c619de tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:21.529924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-40830f7f-48d3-4144-917a-3501dcb5605e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:21.531074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-40830f7f-48d3-4144-917a-3501dcb5605e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:21.544279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40830f7f-48d3-4144-917a-3501dcb5605e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:21.545183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40830f7f-48d3-4144-917a-3501dcb5605e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:21.545944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40830f7f-48d3-4144-917a-3501dcb5605e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:21.547378 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40830f7f-48d3-4144-917a-3501dcb5605e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:21.548073 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40830f7f-48d3-4144-917a-3501dcb5605e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:21.548897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40830f7f-48d3-4144-917a-3501dcb5605e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:21.634170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6c32a815-08d4-4ff4-b7aa-eb52e1c619de tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:21.654918 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6c32a815-08d4-4ff4-b7aa-eb52e1c619de tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1505 microversion: 2.1 time: 0.260754 Aug 07 01:12:21.657152 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 123/249] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:21 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1505 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:21.668832 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0d496dd6-0448-4896-8b02-2114ececf7a3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:21.669715 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0d496dd6-0448-4896-8b02-2114ececf7a3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:21.677522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d496dd6-0448-4896-8b02-2114ececf7a3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:21.678200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d496dd6-0448-4896-8b02-2114ececf7a3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:21.678988 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d496dd6-0448-4896-8b02-2114ececf7a3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:21.679374 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d496dd6-0448-4896-8b02-2114ececf7a3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:21.679835 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d496dd6-0448-4896-8b02-2114ececf7a3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:21.680292 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d496dd6-0448-4896-8b02-2114ececf7a3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:21.796478 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0d496dd6-0448-4896-8b02-2114ececf7a3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:21.810703 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0d496dd6-0448-4896-8b02-2114ececf7a3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1219 microversion: 2.1 time: 0.151730 Aug 07 01:12:21.811618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 124/250] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:21 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1219 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:21.814171 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-40830f7f-48d3-4144-917a-3501dcb5605e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:21.829495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-40830f7f-48d3-4144-917a-3501dcb5605e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:21.832197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-40830f7f-48d3-4144-917a-3501dcb5605e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:21.833665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-40830f7f-48d3-4144-917a-3501dcb5605e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.311814 Aug 07 01:12:21.834441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 127/251] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:21 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:21.851905 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0ccd2249-4868-4030-805a-3886add6dc28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:21.853321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0ccd2249-4868-4030-805a-3886add6dc28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:21.865019 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ccd2249-4868-4030-805a-3886add6dc28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:21.866165 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ccd2249-4868-4030-805a-3886add6dc28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:21.866956 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ccd2249-4868-4030-805a-3886add6dc28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:21.868633 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ccd2249-4868-4030-805a-3886add6dc28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:21.869473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ccd2249-4868-4030-805a-3886add6dc28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:21.870234 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ccd2249-4868-4030-805a-3886add6dc28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:21.948384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ceb2c2b5-b994-4c12-b2aa-bffea010ee25 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:21.949517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ceb2c2b5-b994-4c12-b2aa-bffea010ee25 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:21.961397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ceb2c2b5-b994-4c12-b2aa-bffea010ee25 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:21.962279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ceb2c2b5-b994-4c12-b2aa-bffea010ee25 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:21.963356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ceb2c2b5-b994-4c12-b2aa-bffea010ee25 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:21.965009 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ceb2c2b5-b994-4c12-b2aa-bffea010ee25 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:21.966181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ceb2c2b5-b994-4c12-b2aa-bffea010ee25 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:21.967375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ceb2c2b5-b994-4c12-b2aa-bffea010ee25 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:22.063065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0ccd2249-4868-4030-805a-3886add6dc28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:22.073151 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ceb2c2b5-b994-4c12-b2aa-bffea010ee25 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:22.075700 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0ccd2249-4868-4030-805a-3886add6dc28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.230483 Aug 07 01:12:22.076649 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 125/252] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:21 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:22.088866 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ceb2c2b5-b994-4c12-b2aa-bffea010ee25 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1246 microversion: 2.1 time: 0.148996 Aug 07 01:12:22.090722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 128/253] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:21 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1246 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:22.697689 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-651e8a8e-9927-40ff-b5b3-a6354a731403 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:22.698495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-651e8a8e-9927-40ff-b5b3-a6354a731403 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:22.706175 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-651e8a8e-9927-40ff-b5b3-a6354a731403 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:22.711491 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-651e8a8e-9927-40ff-b5b3-a6354a731403 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:22.712829 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-651e8a8e-9927-40ff-b5b3-a6354a731403 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:22.716676 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-651e8a8e-9927-40ff-b5b3-a6354a731403 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:22.717464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-651e8a8e-9927-40ff-b5b3-a6354a731403 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:22.719430 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-651e8a8e-9927-40ff-b5b3-a6354a731403 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:22.859768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-149c5028-0d6d-4658-8cf6-5b1ca54f3ad3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:22.861028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-149c5028-0d6d-4658-8cf6-5b1ca54f3ad3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:22.900766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-149c5028-0d6d-4658-8cf6-5b1ca54f3ad3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:22.903283 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-149c5028-0d6d-4658-8cf6-5b1ca54f3ad3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:22.904875 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-149c5028-0d6d-4658-8cf6-5b1ca54f3ad3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:22.906507 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-149c5028-0d6d-4658-8cf6-5b1ca54f3ad3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:22.909932 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-149c5028-0d6d-4658-8cf6-5b1ca54f3ad3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:22.913023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-149c5028-0d6d-4658-8cf6-5b1ca54f3ad3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:23.170723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=4 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:23.223824 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-651e8a8e-9927-40ff-b5b3-a6354a731403 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:23.255453 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-651e8a8e-9927-40ff-b5b3-a6354a731403 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1505 microversion: 2.1 time: 0.561356 Aug 07 01:12:23.259213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 126/254] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:22 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1505 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:23.271992 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2f506278-30bb-4d0d-9bcc-5bc96b194765 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:23.276419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2f506278-30bb-4d0d-9bcc-5bc96b194765 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:23.287732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f506278-30bb-4d0d-9bcc-5bc96b194765 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:23.288743 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f506278-30bb-4d0d-9bcc-5bc96b194765 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:23.289382 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f506278-30bb-4d0d-9bcc-5bc96b194765 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:23.290392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f506278-30bb-4d0d-9bcc-5bc96b194765 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:23.291063 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f506278-30bb-4d0d-9bcc-5bc96b194765 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:23.291902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f506278-30bb-4d0d-9bcc-5bc96b194765 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:23.496957 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=30 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:23.572860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-149c5028-0d6d-4658-8cf6-5b1ca54f3ad3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:23.587409 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-149c5028-0d6d-4658-8cf6-5b1ca54f3ad3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1211 microversion: 2.1 time: 0.746054 Aug 07 01:12:23.588550 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 129/255] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:22 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1211 bytes in 748 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:23.598621 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ad1fde25-11dc-41de-a392-f65bb101ebd5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:23.600992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ad1fde25-11dc-41de-a392-f65bb101ebd5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:23.621772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2f506278-30bb-4d0d-9bcc-5bc96b194765 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:23.632329 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ad1fde25-11dc-41de-a392-f65bb101ebd5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:23.633125 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2f506278-30bb-4d0d-9bcc-5bc96b194765 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:23.633450 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ad1fde25-11dc-41de-a392-f65bb101ebd5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:23.634325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ad1fde25-11dc-41de-a392-f65bb101ebd5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:23.634656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2f506278-30bb-4d0d-9bcc-5bc96b194765 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:23.635808 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ad1fde25-11dc-41de-a392-f65bb101ebd5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:23.635952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2f506278-30bb-4d0d-9bcc-5bc96b194765 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1744 microversion: 2.20 time: 0.374352 Aug 07 01:12:23.636872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ad1fde25-11dc-41de-a392-f65bb101ebd5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:23.637620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 127/256] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:23 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1744 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:23.637942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ad1fde25-11dc-41de-a392-f65bb101ebd5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:23.643858 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-20875790-5cd8-4869-8e3b-e3eaff04361f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:23.644511 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-20875790-5cd8-4869-8e3b-e3eaff04361f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:23.653948 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-20875790-5cd8-4869-8e3b-e3eaff04361f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:23.654472 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-20875790-5cd8-4869-8e3b-e3eaff04361f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:23.654921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-20875790-5cd8-4869-8e3b-e3eaff04361f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:23.655693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-20875790-5cd8-4869-8e3b-e3eaff04361f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:23.656154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-20875790-5cd8-4869-8e3b-e3eaff04361f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:23.656639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-20875790-5cd8-4869-8e3b-e3eaff04361f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:23.772810 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-20875790-5cd8-4869-8e3b-e3eaff04361f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:23.787334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-20875790-5cd8-4869-8e3b-e3eaff04361f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1294 microversion: 2.1 time: 0.147796 Aug 07 01:12:23.788088 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 128/257] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:23 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1294 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:24.261780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ad1fde25-11dc-41de-a392-f65bb101ebd5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:24.297142 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1eeb7180-1dd6-44ee-b373-88263b27b2d1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:24.297942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ad1fde25-11dc-41de-a392-f65bb101ebd5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.707136 Aug 07 01:12:24.298713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 130/258] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:23 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 709 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:24.302241 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1eeb7180-1dd6-44ee-b373-88263b27b2d1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:24.320956 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1eeb7180-1dd6-44ee-b373-88263b27b2d1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:24.323306 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1eeb7180-1dd6-44ee-b373-88263b27b2d1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:24.324423 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1eeb7180-1dd6-44ee-b373-88263b27b2d1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:24.325675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1eeb7180-1dd6-44ee-b373-88263b27b2d1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:24.326703 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1eeb7180-1dd6-44ee-b373-88263b27b2d1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:24.327592 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1eeb7180-1dd6-44ee-b373-88263b27b2d1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:24.598988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-089072d4-fac1-4f95-bd34-9a2b4dd41569 req-f7ae649d-de8e-4be3-9f2d-231ed0819091 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "95d224a0-7a33-422a-bae6-06f4acfac258", "name": "network-vif-deleted", "tag": "0108d266-48bb-4aec-97ba-3cbd0446ef81"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:24.638148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1eeb7180-1dd6-44ee-b373-88263b27b2d1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dfc9712739724c91bd9dd5fdaaebd642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1c3de7a94e61473aa759f03c19119775', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:24.641039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-089072d4-fac1-4f95-bd34-9a2b4dd41569 req-f7ae649d-de8e-4be3-9f2d-231ed0819091 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:24.641919 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-089072d4-fac1-4f95-bd34-9a2b4dd41569 req-f7ae649d-de8e-4be3-9f2d-231ed0819091 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:24.642587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-089072d4-fac1-4f95-bd34-9a2b4dd41569 req-f7ae649d-de8e-4be3-9f2d-231ed0819091 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:24.657704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-089072d4-fac1-4f95-bd34-9a2b4dd41569 req-f7ae649d-de8e-4be3-9f2d-231ed0819091 service nova] Creating event network-vif-deleted:0108d266-48bb-4aec-97ba-3cbd0446ef81 for instance 95d224a0-7a33-422a-bae6-06f4acfac258 on hv-cinder-maste Aug 07 01:12:24.662064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1eeb7180-1dd6-44ee-b373-88263b27b2d1 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 200 len: 1505 microversion: 2.1 time: 0.374580 Aug 07 01:12:24.667512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-089072d4-fac1-4f95-bd34-9a2b4dd41569 req-f7ae649d-de8e-4be3-9f2d-231ed0819091 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074037 Aug 07 01:12:24.667690 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 129/259] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:24 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 1505 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:24.667831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 131/260] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:12:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:12:24.682929 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-430d73ba-5b7b-4537-8bdb-d871b2b1ac9d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:24.683811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-430d73ba-5b7b-4537-8bdb-d871b2b1ac9d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:24.687579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-09338c47-fd36-4de6-b27d-b050f8af3bc9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:24.688333 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-09338c47-fd36-4de6-b27d-b050f8af3bc9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:24.698145 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09338c47-fd36-4de6-b27d-b050f8af3bc9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:24.698600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09338c47-fd36-4de6-b27d-b050f8af3bc9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:24.699047 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09338c47-fd36-4de6-b27d-b050f8af3bc9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:24.699772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09338c47-fd36-4de6-b27d-b050f8af3bc9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:24.700524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09338c47-fd36-4de6-b27d-b050f8af3bc9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:24.701047 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09338c47-fd36-4de6-b27d-b050f8af3bc9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:24.703471 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-430d73ba-5b7b-4537-8bdb-d871b2b1ac9d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:24.705452 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-430d73ba-5b7b-4537-8bdb-d871b2b1ac9d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:24.706612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-430d73ba-5b7b-4537-8bdb-d871b2b1ac9d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:24.708614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-430d73ba-5b7b-4537-8bdb-d871b2b1ac9d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:24.709128 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-430d73ba-5b7b-4537-8bdb-d871b2b1ac9d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:24.709586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-430d73ba-5b7b-4537-8bdb-d871b2b1ac9d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:24.827814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-09338c47-fd36-4de6-b27d-b050f8af3bc9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:24.842690 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-09338c47-fd36-4de6-b27d-b050f8af3bc9 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1267 microversion: 2.1 time: 0.169034 Aug 07 01:12:24.844080 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 130/261] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:24 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1267 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:24.852471 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-be4a2536-d886-4600-82aa-37fd2891c09d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:24.854244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-be4a2536-d886-4600-82aa-37fd2891c09d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:24.865538 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be4a2536-d886-4600-82aa-37fd2891c09d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:24.866464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be4a2536-d886-4600-82aa-37fd2891c09d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:24.867185 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be4a2536-d886-4600-82aa-37fd2891c09d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:24.868356 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be4a2536-d886-4600-82aa-37fd2891c09d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:24.869010 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be4a2536-d886-4600-82aa-37fd2891c09d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:24.869544 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be4a2536-d886-4600-82aa-37fd2891c09d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:24.951778 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-be4a2536-d886-4600-82aa-37fd2891c09d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:24.966377 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-be4a2536-d886-4600-82aa-37fd2891c09d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1302 microversion: 2.1 time: 0.119909 Aug 07 01:12:24.967245 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 131/262] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:24 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1302 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:25.001616 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-430d73ba-5b7b-4537-8bdb-d871b2b1ac9d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:25.030326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-430d73ba-5b7b-4537-8bdb-d871b2b1ac9d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:25.043880 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-430d73ba-5b7b-4537-8bdb-d871b2b1ac9d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:25.048842 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-430d73ba-5b7b-4537-8bdb-d871b2b1ac9d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1744 microversion: 2.20 time: 0.382277 Aug 07 01:12:25.051034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 132/263] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:24 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1744 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:25.322980 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-631abc76-8ec2-4d19-9ba2-5ab7addddf1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:25.324428 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-631abc76-8ec2-4d19-9ba2-5ab7addddf1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:25.347508 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-631abc76-8ec2-4d19-9ba2-5ab7addddf1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:25.348416 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-631abc76-8ec2-4d19-9ba2-5ab7addddf1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:25.349133 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-631abc76-8ec2-4d19-9ba2-5ab7addddf1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:25.350226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-631abc76-8ec2-4d19-9ba2-5ab7addddf1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:25.354361 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-631abc76-8ec2-4d19-9ba2-5ab7addddf1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:25.355184 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-631abc76-8ec2-4d19-9ba2-5ab7addddf1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:25.370234 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-944cf8e7-ea2f-4091-aaa6-4651832435c9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:25.385014 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-944cf8e7-ea2f-4091-aaa6-4651832435c9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:25.387302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-944cf8e7-ea2f-4091-aaa6-4651832435c9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:25.388301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-944cf8e7-ea2f-4091-aaa6-4651832435c9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:25.549821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-944cf8e7-ea2f-4091-aaa6-4651832435c9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Going to try to stop instance {{(pid=117983) force_stop /opt/stack/nova/nova/compute/api.py:2709}} Aug 07 01:12:25.616440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-944cf8e7-ea2f-4091-aaa6-4651832435c9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "POST /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/action" status: 202 len: 0 microversion: 2.1 time: 0.254751 Aug 07 01:12:25.618061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 133/264] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:12:25 2022] POST /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/action => generated 0 bytes in 258 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 07 01:12:25.633897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-29d7b60c-e88a-4c88-b08c-660cb651253e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:25.635469 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-29d7b60c-e88a-4c88-b08c-660cb651253e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:25.647936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-29d7b60c-e88a-4c88-b08c-660cb651253e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:25.648563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-29d7b60c-e88a-4c88-b08c-660cb651253e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:25.649166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-29d7b60c-e88a-4c88-b08c-660cb651253e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:25.649961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-29d7b60c-e88a-4c88-b08c-660cb651253e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:25.650671 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-29d7b60c-e88a-4c88-b08c-660cb651253e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:25.651325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-29d7b60c-e88a-4c88-b08c-660cb651253e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:26.301193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-631abc76-8ec2-4d19-9ba2-5ab7addddf1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:26.338896 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-631abc76-8ec2-4d19-9ba2-5ab7addddf1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 1.018681 Aug 07 01:12:26.350514 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 132/265] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:25 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 1028 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:26.370602 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d59a31bb-7212-4901-9694-08a559d280f3 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:26.380989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d59a31bb-7212-4901-9694-08a559d280f3 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] [instance: 95d224a0-7a33-422a-bae6-06f4acfac258] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:26.400843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d59a31bb-7212-4901-9694-08a559d280f3 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:26.407569 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d59a31bb-7212-4901-9694-08a559d280f3 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:26.411920 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d59a31bb-7212-4901-9694-08a559d280f3 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:26.415415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d59a31bb-7212-4901-9694-08a559d280f3 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:26.419739 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d59a31bb-7212-4901-9694-08a559d280f3 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:26.425846 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d59a31bb-7212-4901-9694-08a559d280f3 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:26.498875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-d59a31bb-7212-4901-9694-08a559d280f3 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] HTTP exception thrown: Instance 95d224a0-7a33-422a-bae6-06f4acfac258 could not be found. Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d59a31bb-7212-4901-9694-08a559d280f3 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Request method failure captured: Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 HTTP/1.1 Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xEvnCKhv_-6LE6Um031urOZQaAAt-YamYCb7HhKLY-mIhVKVo1ZjqLKAq6gmY3LtPMj9CUh7oZEpIH7ELdw9Vb0bAx5P3NnhUBTMcy1-w4OCPMLhtRXUP_3Mh8dOWviq83wPHfFsxdB4XdDahM2XmCR8uFR4Z0dBTeqXTQbjVpMer0 Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 1c3de7a94e61473aa759f03c19119775 Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-VolumesActionsTest-556910903 Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:12:26.502036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:12:26.503365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c3de7a94e61473aa759f03c19119775"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1c3de7a94e61473aa759f03c19119775"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1c3de7a94e61473aa759f03c19119775"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1c3de7a94e61473aa759f03c19119775"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:12:26.503365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-VolumesActionsTest-556910903 Aug 07 01:12:26.503365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 1c3de7a94e61473aa759f03c19119775 Aug 07 01:12:26.503365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-VolumesActionsTest-556910903 Aug 07 01:12:26.503365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-VolumesActionsTest-556910903-project Aug 07 01:12:26.503365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:12:26.503365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:12:26.503365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: dfc9712739724c91bd9dd5fdaaebd642 Aug 07 01:12:26.503365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-VolumesActionsTest-556910903-project Aug 07 01:12:26.503365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:12:26.503365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 95d224a0-7a33-422a-bae6-06f4acfac258 could not be found. Aug 07 01:12:26.503365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '95d224a0-7a33-422a-bae6-06f4acfac258'} Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 95d224a0-7a33-422a-bae6-06f4acfac258 could not be found. Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:12:26.504026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 95d224a0-7a33-422a-bae6-06f4acfac258 could not be found. Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 95d224a0-7a33-422a-bae6-06f4acfac258 could not be found. Aug 07 01:12:26.505203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:12:26.506377 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d59a31bb-7212-4901-9694-08a559d280f3 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] Returning 404 to user: Instance 95d224a0-7a33-422a-bae6-06f4acfac258 could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:12:26.506377 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d59a31bb-7212-4901-9694-08a559d280f3 tempest-VolumesActionsTest-556910903 tempest-VolumesActionsTest-556910903-project] 10.222.0.26 "GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258" status: 404 len: 111 microversion: 2.1 time: 0.151775 Aug 07 01:12:26.511827 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 133/266] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:26 2022] GET /compute/v2.1/servers/95d224a0-7a33-422a-bae6-06f4acfac258 => generated 111 bytes in 160 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:12:26.515832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-29d7b60c-e88a-4c88-b08c-660cb651253e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:26.537393 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2df7cfee-10fc-404b-ad6c-4dd762082f8d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:26.548455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-29d7b60c-e88a-4c88-b08c-660cb651253e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1724 microversion: 2.1 time: 0.922960 Aug 07 01:12:26.553898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 134/267] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:25 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1724 bytes in 931 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:26.557649 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2df7cfee-10fc-404b-ad6c-4dd762082f8d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:26.588395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b80364b2-4dff-4445-bf21-b3796e383558 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:26.596486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b80364b2-4dff-4445-bf21-b3796e383558 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:26.601583 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2df7cfee-10fc-404b-ad6c-4dd762082f8d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:26.604555 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2df7cfee-10fc-404b-ad6c-4dd762082f8d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:26.605565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2df7cfee-10fc-404b-ad6c-4dd762082f8d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:26.607904 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2df7cfee-10fc-404b-ad6c-4dd762082f8d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:26.609673 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2df7cfee-10fc-404b-ad6c-4dd762082f8d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:26.613102 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2df7cfee-10fc-404b-ad6c-4dd762082f8d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:26.623634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b80364b2-4dff-4445-bf21-b3796e383558 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:26.624559 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b80364b2-4dff-4445-bf21-b3796e383558 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:26.625301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b80364b2-4dff-4445-bf21-b3796e383558 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:26.626460 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b80364b2-4dff-4445-bf21-b3796e383558 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:26.627866 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b80364b2-4dff-4445-bf21-b3796e383558 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:26.628604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b80364b2-4dff-4445-bf21-b3796e383558 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:26.805420 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2df7cfee-10fc-404b-ad6c-4dd762082f8d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:26.838406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2df7cfee-10fc-404b-ad6c-4dd762082f8d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1273 microversion: 2.1 time: 0.320484 Aug 07 01:12:26.852560 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 134/268] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:26 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1273 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:26.866708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1269c96b-77ad-4817-b199-1751429892ce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:26.868045 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1269c96b-77ad-4817-b199-1751429892ce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:26.899948 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1269c96b-77ad-4817-b199-1751429892ce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:26.902040 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1269c96b-77ad-4817-b199-1751429892ce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:26.902929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1269c96b-77ad-4817-b199-1751429892ce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:26.910283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1269c96b-77ad-4817-b199-1751429892ce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:26.911159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1269c96b-77ad-4817-b199-1751429892ce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:26.912268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1269c96b-77ad-4817-b199-1751429892ce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:26.956798 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b80364b2-4dff-4445-bf21-b3796e383558 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:26.978687 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b80364b2-4dff-4445-bf21-b3796e383558 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1300 microversion: 2.1 time: 0.422421 Aug 07 01:12:26.984604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 135/269] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:26 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1300 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:27.386529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-563bc8c1-ca16-402c-aee3-f3a4dee54039 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:27.387697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-563bc8c1-ca16-402c-aee3-f3a4dee54039 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:27.425683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-563bc8c1-ca16-402c-aee3-f3a4dee54039 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:27.435170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-563bc8c1-ca16-402c-aee3-f3a4dee54039 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:27.436408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-563bc8c1-ca16-402c-aee3-f3a4dee54039 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:27.442130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-563bc8c1-ca16-402c-aee3-f3a4dee54039 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:27.443488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-563bc8c1-ca16-402c-aee3-f3a4dee54039 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:27.444853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-563bc8c1-ca16-402c-aee3-f3a4dee54039 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:27.792700 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1269c96b-77ad-4817-b199-1751429892ce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:27.834413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1269c96b-77ad-4817-b199-1751429892ce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:27.854769 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1269c96b-77ad-4817-b199-1751429892ce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:27.865186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1269c96b-77ad-4817-b199-1751429892ce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1739 microversion: 2.20 time: 1.007641 Aug 07 01:12:27.866463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 135/270] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:26 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1739 bytes in 1013 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:27.896280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9065ba94-b7a4-4fe3-a939-bc11c80f6e91 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:27.896280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9065ba94-b7a4-4fe3-a939-bc11c80f6e91 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:27.904189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-563bc8c1-ca16-402c-aee3-f3a4dee54039 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:27.919236 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9065ba94-b7a4-4fe3-a939-bc11c80f6e91 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:27.923692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9065ba94-b7a4-4fe3-a939-bc11c80f6e91 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:27.925748 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9065ba94-b7a4-4fe3-a939-bc11c80f6e91 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:27.930352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9065ba94-b7a4-4fe3-a939-bc11c80f6e91 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:27.931701 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9065ba94-b7a4-4fe3-a939-bc11c80f6e91 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:27.935570 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-563bc8c1-ca16-402c-aee3-f3a4dee54039 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.570997 Aug 07 01:12:27.936103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9065ba94-b7a4-4fe3-a939-bc11c80f6e91 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:27.943123 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 136/271] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:27 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 580 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:27.963468 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-77cdfc49-0692-473b-8568-fe30f9115ffd tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:27.971571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-77cdfc49-0692-473b-8568-fe30f9115ffd tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:28.000423 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77cdfc49-0692-473b-8568-fe30f9115ffd tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:28.001626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77cdfc49-0692-473b-8568-fe30f9115ffd tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:28.002412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77cdfc49-0692-473b-8568-fe30f9115ffd tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:28.003423 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77cdfc49-0692-473b-8568-fe30f9115ffd tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:28.005658 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77cdfc49-0692-473b-8568-fe30f9115ffd tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:28.009078 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77cdfc49-0692-473b-8568-fe30f9115ffd tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:28.020750 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:28.831961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9065ba94-b7a4-4fe3-a939-bc11c80f6e91 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:28.862780 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9065ba94-b7a4-4fe3-a939-bc11c80f6e91 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1724 microversion: 2.1 time: 0.992595 Aug 07 01:12:28.867113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 136/272] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:27 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1724 bytes in 1000 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:28.889231 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9eec414a-46ee-4812-8ccb-2f16f6133dda tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:28.890663 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9eec414a-46ee-4812-8ccb-2f16f6133dda tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:28.913570 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9eec414a-46ee-4812-8ccb-2f16f6133dda tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:28.923150 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9eec414a-46ee-4812-8ccb-2f16f6133dda tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:28.924704 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9eec414a-46ee-4812-8ccb-2f16f6133dda tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:28.926816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9eec414a-46ee-4812-8ccb-2f16f6133dda tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:28.928191 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9eec414a-46ee-4812-8ccb-2f16f6133dda tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:28.931608 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9eec414a-46ee-4812-8ccb-2f16f6133dda tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:29.030274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-77cdfc49-0692-473b-8568-fe30f9115ffd tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:29.054818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-77cdfc49-0692-473b-8568-fe30f9115ffd tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1273 microversion: 2.1 time: 1.109046 Aug 07 01:12:29.055666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 137/273] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:27 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1273 bytes in 1112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:29.084293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b1861074-8dff-4324-abfd-188730f6cebb tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:29.084293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b1861074-8dff-4324-abfd-188730f6cebb tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:29.101751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b1861074-8dff-4324-abfd-188730f6cebb tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:29.102397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b1861074-8dff-4324-abfd-188730f6cebb tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:29.102909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b1861074-8dff-4324-abfd-188730f6cebb tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:29.103766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b1861074-8dff-4324-abfd-188730f6cebb tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:29.104269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b1861074-8dff-4324-abfd-188730f6cebb tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:29.111637 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b1861074-8dff-4324-abfd-188730f6cebb tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:29.651731 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9eec414a-46ee-4812-8ccb-2f16f6133dda tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:29.668169 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b1861074-8dff-4324-abfd-188730f6cebb tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:29.696662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9eec414a-46ee-4812-8ccb-2f16f6133dda tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.827090 Aug 07 01:12:29.723011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b1861074-8dff-4324-abfd-188730f6cebb tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:29.730364 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 137/274] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:28 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 862 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:29.739590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b1861074-8dff-4324-abfd-188730f6cebb tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:29.743102 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b1861074-8dff-4324-abfd-188730f6cebb tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1739 microversion: 2.20 time: 0.684368 Aug 07 01:12:29.744095 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 138/275] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:29 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1739 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:29.746152 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4674e2de-5ab0-4588-87d0-591738914827 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:29.750085 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4674e2de-5ab0-4588-87d0-591738914827 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:29.766360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-a05570dc-8dba-4bc7-9c8c-c92dd072220b req-9999cfe7-8ba1-496d-ac51-48584e735b82 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7f60fb22-d17a-4b7c-9b7d-95543be1907e", "name": "network-vif-plugged", "status": "completed", "tag": "9ab59ce3-ed0d-4a8e-b76b-adeff559725a"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:29.777245 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4674e2de-5ab0-4588-87d0-591738914827 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:29.778671 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4674e2de-5ab0-4588-87d0-591738914827 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:29.784634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4674e2de-5ab0-4588-87d0-591738914827 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:29.788809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4674e2de-5ab0-4588-87d0-591738914827 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:29.791780 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4674e2de-5ab0-4588-87d0-591738914827 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:29.794914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4674e2de-5ab0-4588-87d0-591738914827 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:29.814572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a05570dc-8dba-4bc7-9c8c-c92dd072220b req-9999cfe7-8ba1-496d-ac51-48584e735b82 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:29.820434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a05570dc-8dba-4bc7-9c8c-c92dd072220b req-9999cfe7-8ba1-496d-ac51-48584e735b82 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:29.821769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a05570dc-8dba-4bc7-9c8c-c92dd072220b req-9999cfe7-8ba1-496d-ac51-48584e735b82 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:29.866055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-a05570dc-8dba-4bc7-9c8c-c92dd072220b req-9999cfe7-8ba1-496d-ac51-48584e735b82 service nova] Creating event network-vif-plugged:9ab59ce3-ed0d-4a8e-b76b-adeff559725a for instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e on hv-cinder-maste Aug 07 01:12:29.895452 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-a05570dc-8dba-4bc7-9c8c-c92dd072220b req-9999cfe7-8ba1-496d-ac51-48584e735b82 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.140288 Aug 07 01:12:29.896597 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 139/276] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:12:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:12:29.914986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7d77a705-b0b9-4b04-ba94-ff4cf2a5c29e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:29.916291 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7d77a705-b0b9-4b04-ba94-ff4cf2a5c29e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:29.950957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d77a705-b0b9-4b04-ba94-ff4cf2a5c29e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:29.956178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d77a705-b0b9-4b04-ba94-ff4cf2a5c29e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:29.958018 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d77a705-b0b9-4b04-ba94-ff4cf2a5c29e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:29.961656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d77a705-b0b9-4b04-ba94-ff4cf2a5c29e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:29.968205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d77a705-b0b9-4b04-ba94-ff4cf2a5c29e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:29.971510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d77a705-b0b9-4b04-ba94-ff4cf2a5c29e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:30.293707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4674e2de-5ab0-4588-87d0-591738914827 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:30.316294 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4674e2de-5ab0-4588-87d0-591738914827 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.584078 Aug 07 01:12:30.322319 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 138/277] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:29 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:30.335632 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c8ae2d05-aa63-4e5c-b21f-33807ac72e13 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:30.343768 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c8ae2d05-aa63-4e5c-b21f-33807ac72e13 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:30.358386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c8ae2d05-aa63-4e5c-b21f-33807ac72e13 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:30.359280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c8ae2d05-aa63-4e5c-b21f-33807ac72e13 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:30.360106 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c8ae2d05-aa63-4e5c-b21f-33807ac72e13 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:30.362732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c8ae2d05-aa63-4e5c-b21f-33807ac72e13 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:30.363520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c8ae2d05-aa63-4e5c-b21f-33807ac72e13 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:30.364270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c8ae2d05-aa63-4e5c-b21f-33807ac72e13 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:30.500271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7d77a705-b0b9-4b04-ba94-ff4cf2a5c29e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:30.539982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7d77a705-b0b9-4b04-ba94-ff4cf2a5c29e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1724 microversion: 2.1 time: 0.635556 Aug 07 01:12:30.566080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 140/278] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:29 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1724 bytes in 651 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:30.738706 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-67943f12-e7e2-4d1a-b39f-5f719db97e2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:30.739825 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-67943f12-e7e2-4d1a-b39f-5f719db97e2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:30.747074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67943f12-e7e2-4d1a-b39f-5f719db97e2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:30.747663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67943f12-e7e2-4d1a-b39f-5f719db97e2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:30.748201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67943f12-e7e2-4d1a-b39f-5f719db97e2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:30.749002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67943f12-e7e2-4d1a-b39f-5f719db97e2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:30.749708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67943f12-e7e2-4d1a-b39f-5f719db97e2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:30.750474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67943f12-e7e2-4d1a-b39f-5f719db97e2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:30.752115 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c8ae2d05-aa63-4e5c-b21f-33807ac72e13 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:30.767774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c8ae2d05-aa63-4e5c-b21f-33807ac72e13 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1315 microversion: 2.1 time: 0.444394 Aug 07 01:12:30.769757 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 139/279] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:30 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1315 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:30.778703 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-47d1a132-b662-4708-81fd-d561b5d66aff tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:30.779881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-47d1a132-b662-4708-81fd-d561b5d66aff tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:30.791500 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-47d1a132-b662-4708-81fd-d561b5d66aff tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:30.792445 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-47d1a132-b662-4708-81fd-d561b5d66aff tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:30.793207 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-47d1a132-b662-4708-81fd-d561b5d66aff tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:30.794871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-47d1a132-b662-4708-81fd-d561b5d66aff tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:30.796050 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-47d1a132-b662-4708-81fd-d561b5d66aff tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:30.797146 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-47d1a132-b662-4708-81fd-d561b5d66aff tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:30.997485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-67943f12-e7e2-4d1a-b39f-5f719db97e2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:31.012354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-67943f12-e7e2-4d1a-b39f-5f719db97e2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.277855 Aug 07 01:12:31.029039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 141/280] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:30 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:31.091305 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-47d1a132-b662-4708-81fd-d561b5d66aff tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:31.108444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-47d1a132-b662-4708-81fd-d561b5d66aff tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:31.112681 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-47d1a132-b662-4708-81fd-d561b5d66aff tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:31.115603 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-47d1a132-b662-4708-81fd-d561b5d66aff tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1739 microversion: 2.20 time: 0.344124 Aug 07 01:12:31.134020 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 140/281] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:30 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1739 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:31.359170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ba3855d5-94c7-4858-a6a8-e109afa690fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:31.362622 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ba3855d5-94c7-4858-a6a8-e109afa690fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:31.383362 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba3855d5-94c7-4858-a6a8-e109afa690fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:31.387782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba3855d5-94c7-4858-a6a8-e109afa690fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:31.390806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba3855d5-94c7-4858-a6a8-e109afa690fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:31.394289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba3855d5-94c7-4858-a6a8-e109afa690fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:31.395661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba3855d5-94c7-4858-a6a8-e109afa690fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:31.396873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba3855d5-94c7-4858-a6a8-e109afa690fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:31.461531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:31.594083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-eb91956e-7b01-4fae-86aa-a9e90b80ac54 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:31.597238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-eb91956e-7b01-4fae-86aa-a9e90b80ac54 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:31.609659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb91956e-7b01-4fae-86aa-a9e90b80ac54 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:31.611097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb91956e-7b01-4fae-86aa-a9e90b80ac54 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:31.612898 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb91956e-7b01-4fae-86aa-a9e90b80ac54 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:31.614779 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb91956e-7b01-4fae-86aa-a9e90b80ac54 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:31.616192 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb91956e-7b01-4fae-86aa-a9e90b80ac54 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:31.617784 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb91956e-7b01-4fae-86aa-a9e90b80ac54 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:31.733059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ba3855d5-94c7-4858-a6a8-e109afa690fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:31.749096 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ba3855d5-94c7-4858-a6a8-e109afa690fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.409390 Aug 07 01:12:31.749998 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 142/282] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:31 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:31.816315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-caba65db-ff2d-4206-84f5-f9dd0d102b2e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:31.817091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-caba65db-ff2d-4206-84f5-f9dd0d102b2e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:31.824925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caba65db-ff2d-4206-84f5-f9dd0d102b2e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:31.825740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caba65db-ff2d-4206-84f5-f9dd0d102b2e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:31.826418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caba65db-ff2d-4206-84f5-f9dd0d102b2e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:31.827544 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caba65db-ff2d-4206-84f5-f9dd0d102b2e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:31.828229 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caba65db-ff2d-4206-84f5-f9dd0d102b2e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:31.828884 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caba65db-ff2d-4206-84f5-f9dd0d102b2e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:31.901563 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-eb91956e-7b01-4fae-86aa-a9e90b80ac54 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:31.933652 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-eb91956e-7b01-4fae-86aa-a9e90b80ac54 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1724 microversion: 2.1 time: 0.362034 Aug 07 01:12:31.946351 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 141/283] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:31 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1724 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:32.053037 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-06e5d050-1a5a-4d53-98ea-f692bc742f12 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:32.054310 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-06e5d050-1a5a-4d53-98ea-f692bc742f12 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:32.066278 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06e5d050-1a5a-4d53-98ea-f692bc742f12 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:32.067311 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06e5d050-1a5a-4d53-98ea-f692bc742f12 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:32.067778 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06e5d050-1a5a-4d53-98ea-f692bc742f12 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:32.075431 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06e5d050-1a5a-4d53-98ea-f692bc742f12 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:32.076685 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06e5d050-1a5a-4d53-98ea-f692bc742f12 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:32.077545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06e5d050-1a5a-4d53-98ea-f692bc742f12 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:32.132177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-caba65db-ff2d-4206-84f5-f9dd0d102b2e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:32.144084 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-caba65db-ff2d-4206-84f5-f9dd0d102b2e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1315 microversion: 2.1 time: 0.331512 Aug 07 01:12:32.150539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 143/284] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:31 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1315 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:32.171848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-aa769003-49eb-42bd-87a3-dc17cb860933 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:32.173172 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-aa769003-49eb-42bd-87a3-dc17cb860933 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:32.188822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa769003-49eb-42bd-87a3-dc17cb860933 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:32.190938 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa769003-49eb-42bd-87a3-dc17cb860933 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:32.191783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa769003-49eb-42bd-87a3-dc17cb860933 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:32.193151 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa769003-49eb-42bd-87a3-dc17cb860933 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:32.194244 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa769003-49eb-42bd-87a3-dc17cb860933 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:32.195029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa769003-49eb-42bd-87a3-dc17cb860933 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:32.389130 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-06e5d050-1a5a-4d53-98ea-f692bc742f12 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:32.415773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-06e5d050-1a5a-4d53-98ea-f692bc742f12 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.368147 Aug 07 01:12:32.418641 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 142/285] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:32 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:32.630467 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-aa769003-49eb-42bd-87a3-dc17cb860933 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:32.648940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-aa769003-49eb-42bd-87a3-dc17cb860933 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:32.650990 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-aa769003-49eb-42bd-87a3-dc17cb860933 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:32.657184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-aa769003-49eb-42bd-87a3-dc17cb860933 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1739 microversion: 2.20 time: 0.490027 Aug 07 01:12:32.659719 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 144/286] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:32 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1739 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:32.779852 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-28ebf017-3b92-4db9-b519-6d7347c1c46e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:32.782486 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-28ebf017-3b92-4db9-b519-6d7347c1c46e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:32.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-28ebf017-3b92-4db9-b519-6d7347c1c46e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:32.797418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-28ebf017-3b92-4db9-b519-6d7347c1c46e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:32.798953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-28ebf017-3b92-4db9-b519-6d7347c1c46e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:32.802075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-28ebf017-3b92-4db9-b519-6d7347c1c46e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:32.803775 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-28ebf017-3b92-4db9-b519-6d7347c1c46e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:32.804887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-28ebf017-3b92-4db9-b519-6d7347c1c46e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:32.969696 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d3aa7cdc-b6da-4b1b-9bed-74d25f998ba5 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:32.970354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d3aa7cdc-b6da-4b1b-9bed-74d25f998ba5 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:32.979075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d3aa7cdc-b6da-4b1b-9bed-74d25f998ba5 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:32.979577 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d3aa7cdc-b6da-4b1b-9bed-74d25f998ba5 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:32.980029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d3aa7cdc-b6da-4b1b-9bed-74d25f998ba5 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:32.981545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d3aa7cdc-b6da-4b1b-9bed-74d25f998ba5 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:32.982142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d3aa7cdc-b6da-4b1b-9bed-74d25f998ba5 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:32.983220 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d3aa7cdc-b6da-4b1b-9bed-74d25f998ba5 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:33.365232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d3aa7cdc-b6da-4b1b-9bed-74d25f998ba5 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:33.390407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d3aa7cdc-b6da-4b1b-9bed-74d25f998ba5 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1724 microversion: 2.1 time: 0.426452 Aug 07 01:12:33.391489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 145/287] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:32 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1724 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:33.449517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9a088e44-3430-4909-a717-ecc993194e41 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:33.450771 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9a088e44-3430-4909-a717-ecc993194e41 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:33.468676 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a088e44-3430-4909-a717-ecc993194e41 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:33.481459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a088e44-3430-4909-a717-ecc993194e41 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:33.482991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a088e44-3430-4909-a717-ecc993194e41 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:33.484615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a088e44-3430-4909-a717-ecc993194e41 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:33.485769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a088e44-3430-4909-a717-ecc993194e41 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:33.487406 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a088e44-3430-4909-a717-ecc993194e41 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:33.601493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-28ebf017-3b92-4db9-b519-6d7347c1c46e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:33.654264 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-28ebf017-3b92-4db9-b519-6d7347c1c46e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.885696 Aug 07 01:12:33.663468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 143/288] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:32 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 896 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:33.676268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4e6bcb0b-595e-4025-b7c2-26083e2a68a5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:33.678936 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4e6bcb0b-595e-4025-b7c2-26083e2a68a5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:33.706442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4e6bcb0b-595e-4025-b7c2-26083e2a68a5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:33.707407 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4e6bcb0b-595e-4025-b7c2-26083e2a68a5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:33.708421 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4e6bcb0b-595e-4025-b7c2-26083e2a68a5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:33.709877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4e6bcb0b-595e-4025-b7c2-26083e2a68a5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:33.710885 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4e6bcb0b-595e-4025-b7c2-26083e2a68a5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:33.711785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4e6bcb0b-595e-4025-b7c2-26083e2a68a5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:33.988318 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9a088e44-3430-4909-a717-ecc993194e41 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:34.024175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9a088e44-3430-4909-a717-ecc993194e41 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1315 microversion: 2.1 time: 0.597287 Aug 07 01:12:34.030941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 146/289] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:33 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1315 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:34.047390 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6add23b3-8e32-479a-b5f6-3efffb952312 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:34.049077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6add23b3-8e32-479a-b5f6-3efffb952312 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:34.079359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6add23b3-8e32-479a-b5f6-3efffb952312 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:34.081381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6add23b3-8e32-479a-b5f6-3efffb952312 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:34.082485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6add23b3-8e32-479a-b5f6-3efffb952312 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:34.084085 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6add23b3-8e32-479a-b5f6-3efffb952312 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:34.085291 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6add23b3-8e32-479a-b5f6-3efffb952312 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:34.086342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6add23b3-8e32-479a-b5f6-3efffb952312 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:34.112923 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4e6bcb0b-595e-4025-b7c2-26083e2a68a5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:34.174211 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4e6bcb0b-595e-4025-b7c2-26083e2a68a5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.507880 Aug 07 01:12:34.195334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 144/290] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:33 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:34.448631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d6fd37ee-66a6-4f7f-a996-e1e35327cad6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:34.450906 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d6fd37ee-66a6-4f7f-a996-e1e35327cad6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:34.473636 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d6fd37ee-66a6-4f7f-a996-e1e35327cad6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:34.478745 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d6fd37ee-66a6-4f7f-a996-e1e35327cad6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:34.481400 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d6fd37ee-66a6-4f7f-a996-e1e35327cad6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:34.484177 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d6fd37ee-66a6-4f7f-a996-e1e35327cad6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:34.487856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d6fd37ee-66a6-4f7f-a996-e1e35327cad6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:34.489080 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d6fd37ee-66a6-4f7f-a996-e1e35327cad6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:34.565137 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6add23b3-8e32-479a-b5f6-3efffb952312 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:34.594596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6add23b3-8e32-479a-b5f6-3efffb952312 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:34.598290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6add23b3-8e32-479a-b5f6-3efffb952312 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:34.612738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6add23b3-8e32-479a-b5f6-3efffb952312 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1739 microversion: 2.20 time: 0.567410 Aug 07 01:12:34.614664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 147/291] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:34 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1739 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:34.630612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-f6564cde-f647-4c60-a69e-c445c5eb9f6e req-f81119db-a8f7-488c-b37c-7b96f768f15c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f92846c8-747c-4e27-ba74-21ca0ee12aab", "tag": "fab914e3-36b7-45a0-b7bb-c8529feb8d06"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:34.653325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-f6564cde-f647-4c60-a69e-c445c5eb9f6e req-f81119db-a8f7-488c-b37c-7b96f768f15c service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:34.654191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-f6564cde-f647-4c60-a69e-c445c5eb9f6e req-f81119db-a8f7-488c-b37c-7b96f768f15c service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:34.654902 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-f6564cde-f647-4c60-a69e-c445c5eb9f6e req-f81119db-a8f7-488c-b37c-7b96f768f15c service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:34.686755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-f6564cde-f647-4c60-a69e-c445c5eb9f6e req-f81119db-a8f7-488c-b37c-7b96f768f15c service nova] Creating event network-changed:fab914e3-36b7-45a0-b7bb-c8529feb8d06 for instance f92846c8-747c-4e27-ba74-21ca0ee12aab on hv-cinder-maste Aug 07 01:12:34.701758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-f6564cde-f647-4c60-a69e-c445c5eb9f6e req-f81119db-a8f7-488c-b37c-7b96f768f15c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.084624 Aug 07 01:12:34.703278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 148/292] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:12:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:12:34.714958 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0119300c-f962-4f99-817c-8750b5cba880 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:34.716511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0119300c-f962-4f99-817c-8750b5cba880 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:34.730921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0119300c-f962-4f99-817c-8750b5cba880 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:34.732467 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0119300c-f962-4f99-817c-8750b5cba880 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:34.733689 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0119300c-f962-4f99-817c-8750b5cba880 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:34.735363 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0119300c-f962-4f99-817c-8750b5cba880 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:34.737082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0119300c-f962-4f99-817c-8750b5cba880 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:34.738182 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0119300c-f962-4f99-817c-8750b5cba880 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:35.057376 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d6fd37ee-66a6-4f7f-a996-e1e35327cad6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:35.058636 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0119300c-f962-4f99-817c-8750b5cba880 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:35.079232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0119300c-f962-4f99-817c-8750b5cba880 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.371297 Aug 07 01:12:35.081319 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 149/293] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:34 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:35.085478 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d6fd37ee-66a6-4f7f-a996-e1e35327cad6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1724 microversion: 2.1 time: 0.657233 Aug 07 01:12:35.104016 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-a37f8ce9-e871-47ff-b074-b9751aa299f2 req-6a67e557-54f1-4895-aca0-357851bf07f9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "54a16243-ecd7-4152-a1d3-bf92cc3709cc", "tag": "75592632-690e-4dbf-a132-69141d52e063"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:35.106389 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 145/294] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:34 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1724 bytes in 680 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:35.122791 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d58755cf-54db-4dee-a325-43eb16ac72fc tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:35.123964 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d58755cf-54db-4dee-a325-43eb16ac72fc tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:35.131502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a37f8ce9-e871-47ff-b074-b9751aa299f2 req-6a67e557-54f1-4895-aca0-357851bf07f9 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:35.132456 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a37f8ce9-e871-47ff-b074-b9751aa299f2 req-6a67e557-54f1-4895-aca0-357851bf07f9 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:35.133241 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a37f8ce9-e871-47ff-b074-b9751aa299f2 req-6a67e557-54f1-4895-aca0-357851bf07f9 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:35.135223 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d58755cf-54db-4dee-a325-43eb16ac72fc tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:35.136507 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d58755cf-54db-4dee-a325-43eb16ac72fc tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:35.137555 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d58755cf-54db-4dee-a325-43eb16ac72fc tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:35.139028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d58755cf-54db-4dee-a325-43eb16ac72fc tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:35.140149 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d58755cf-54db-4dee-a325-43eb16ac72fc tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:35.141001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d58755cf-54db-4dee-a325-43eb16ac72fc tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:35.164683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-a37f8ce9-e871-47ff-b074-b9751aa299f2 req-6a67e557-54f1-4895-aca0-357851bf07f9 service nova] Creating event network-changed:75592632-690e-4dbf-a132-69141d52e063 for instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc on hv-cinder-maste Aug 07 01:12:35.177035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-a37f8ce9-e871-47ff-b074-b9751aa299f2 req-6a67e557-54f1-4895-aca0-357851bf07f9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.092625 Aug 07 01:12:35.177464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 150/295] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:12:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:12:35.236136 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dd8da713-489f-4855-81c3-4743e62b56ce tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:35.237703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dd8da713-489f-4855-81c3-4743e62b56ce tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:35.249806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd8da713-489f-4855-81c3-4743e62b56ce tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:35.250862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd8da713-489f-4855-81c3-4743e62b56ce tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:35.251938 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd8da713-489f-4855-81c3-4743e62b56ce tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:35.253705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd8da713-489f-4855-81c3-4743e62b56ce tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:35.254651 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd8da713-489f-4855-81c3-4743e62b56ce tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:35.255533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd8da713-489f-4855-81c3-4743e62b56ce tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:35.418351 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d58755cf-54db-4dee-a325-43eb16ac72fc tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:35.438507 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d58755cf-54db-4dee-a325-43eb16ac72fc tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1315 microversion: 2.1 time: 0.330064 Aug 07 01:12:35.439860 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 146/296] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:35 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1315 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:35.550174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-dd8da713-489f-4855-81c3-4743e62b56ce tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:35.559910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dd8da713-489f-4855-81c3-4743e62b56ce tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.332128 Aug 07 01:12:35.560775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 151/297] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:35 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:35.652279 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1bdfbae1-bc85-4e8c-824c-6d17fd28cd9a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:35.653858 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1bdfbae1-bc85-4e8c-824c-6d17fd28cd9a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:35.666833 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1bdfbae1-bc85-4e8c-824c-6d17fd28cd9a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:35.668364 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1bdfbae1-bc85-4e8c-824c-6d17fd28cd9a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:35.669245 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1bdfbae1-bc85-4e8c-824c-6d17fd28cd9a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:35.670639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1bdfbae1-bc85-4e8c-824c-6d17fd28cd9a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:35.671724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1bdfbae1-bc85-4e8c-824c-6d17fd28cd9a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:35.672806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1bdfbae1-bc85-4e8c-824c-6d17fd28cd9a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:36.121105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cf6d77fe-a35b-4cdb-8dbc-adab85af7eca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:36.127931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cf6d77fe-a35b-4cdb-8dbc-adab85af7eca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:36.161141 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cf6d77fe-a35b-4cdb-8dbc-adab85af7eca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:36.167617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cf6d77fe-a35b-4cdb-8dbc-adab85af7eca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:36.168972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cf6d77fe-a35b-4cdb-8dbc-adab85af7eca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:36.171678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cf6d77fe-a35b-4cdb-8dbc-adab85af7eca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:36.173571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cf6d77fe-a35b-4cdb-8dbc-adab85af7eca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:36.178493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cf6d77fe-a35b-4cdb-8dbc-adab85af7eca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:36.199854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1bdfbae1-bc85-4e8c-824c-6d17fd28cd9a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:36.216164 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1bdfbae1-bc85-4e8c-824c-6d17fd28cd9a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:36.218012 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1bdfbae1-bc85-4e8c-824c-6d17fd28cd9a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:36.219504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1bdfbae1-bc85-4e8c-824c-6d17fd28cd9a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1739 microversion: 2.20 time: 0.575657 Aug 07 01:12:36.224698 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 147/298] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:35 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1739 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:36.233542 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-555457b1-4f4e-43d7-b16b-7134e0a127b6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:36.237859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-555457b1-4f4e-43d7-b16b-7134e0a127b6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:36.257844 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-555457b1-4f4e-43d7-b16b-7134e0a127b6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:36.262530 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-555457b1-4f4e-43d7-b16b-7134e0a127b6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:36.263343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-555457b1-4f4e-43d7-b16b-7134e0a127b6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:36.264533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-555457b1-4f4e-43d7-b16b-7134e0a127b6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:36.265290 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-555457b1-4f4e-43d7-b16b-7134e0a127b6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:36.265946 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-555457b1-4f4e-43d7-b16b-7134e0a127b6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:36.755209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-cf6d77fe-a35b-4cdb-8dbc-adab85af7eca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:36.791274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cf6d77fe-a35b-4cdb-8dbc-adab85af7eca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.671941 Aug 07 01:12:36.792672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 152/299] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:36 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:36.802578 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-079c5eaa-d86c-455b-b16c-bc89a1022026 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:36.803570 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-079c5eaa-d86c-455b-b16c-bc89a1022026 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:36.812269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-079c5eaa-d86c-455b-b16c-bc89a1022026 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:36.813161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-079c5eaa-d86c-455b-b16c-bc89a1022026 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:36.813802 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-079c5eaa-d86c-455b-b16c-bc89a1022026 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:36.816433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-079c5eaa-d86c-455b-b16c-bc89a1022026 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:36.817254 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-079c5eaa-d86c-455b-b16c-bc89a1022026 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:36.818001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-079c5eaa-d86c-455b-b16c-bc89a1022026 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:37.011518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-555457b1-4f4e-43d7-b16b-7134e0a127b6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:37.034882 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-555457b1-4f4e-43d7-b16b-7134e0a127b6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1724 microversion: 2.1 time: 0.808135 Aug 07 01:12:37.036268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 148/300] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:36 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1724 bytes in 811 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:37.052056 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-ef6bc3c1-3ffd-46f2-915f-b6a128fc3deb req-234508e4-a8bc-4b1c-af1e-863cfc331eb2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "99e925dd-a3a5-44ab-bf20-9c80807d5530", "name": "network-vif-plugged", "status": "completed", "tag": "c91dc146-cf64-40b0-a3e9-644405de258c"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:37.085914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-ef6bc3c1-3ffd-46f2-915f-b6a128fc3deb req-234508e4-a8bc-4b1c-af1e-863cfc331eb2 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:37.087024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-ef6bc3c1-3ffd-46f2-915f-b6a128fc3deb req-234508e4-a8bc-4b1c-af1e-863cfc331eb2 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:37.088126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-ef6bc3c1-3ffd-46f2-915f-b6a128fc3deb req-234508e4-a8bc-4b1c-af1e-863cfc331eb2 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:37.130747 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-ef6bc3c1-3ffd-46f2-915f-b6a128fc3deb req-234508e4-a8bc-4b1c-af1e-863cfc331eb2 service nova] Creating event network-vif-plugged:c91dc146-cf64-40b0-a3e9-644405de258c for instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 on hv-cinder-maste Aug 07 01:12:37.141729 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-ef6bc3c1-3ffd-46f2-915f-b6a128fc3deb req-234508e4-a8bc-4b1c-af1e-863cfc331eb2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.103660 Aug 07 01:12:37.142645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 149/301] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:12:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:12:37.154574 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f219ac61-6453-430b-bfcb-8aaefaa5d20e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:37.159714 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f219ac61-6453-430b-bfcb-8aaefaa5d20e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:37.178665 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f219ac61-6453-430b-bfcb-8aaefaa5d20e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:37.179837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f219ac61-6453-430b-bfcb-8aaefaa5d20e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:37.180508 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f219ac61-6453-430b-bfcb-8aaefaa5d20e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:37.185981 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f219ac61-6453-430b-bfcb-8aaefaa5d20e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:37.187225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f219ac61-6453-430b-bfcb-8aaefaa5d20e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:37.191392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f219ac61-6453-430b-bfcb-8aaefaa5d20e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:37.500732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-079c5eaa-d86c-455b-b16c-bc89a1022026 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:37.551647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-079c5eaa-d86c-455b-b16c-bc89a1022026 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1315 microversion: 2.1 time: 0.756104 Aug 07 01:12:37.556786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 153/302] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:36 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1315 bytes in 763 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:37.579128 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-38001036-a6d6-4cd9-8b24-9f47c7bfde90 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:37.584074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-38001036-a6d6-4cd9-8b24-9f47c7bfde90 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:37.620965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38001036-a6d6-4cd9-8b24-9f47c7bfde90 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:37.622810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38001036-a6d6-4cd9-8b24-9f47c7bfde90 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:37.623922 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38001036-a6d6-4cd9-8b24-9f47c7bfde90 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:37.629777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38001036-a6d6-4cd9-8b24-9f47c7bfde90 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:37.633885 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38001036-a6d6-4cd9-8b24-9f47c7bfde90 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:37.636599 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38001036-a6d6-4cd9-8b24-9f47c7bfde90 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:38.716338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f219ac61-6453-430b-bfcb-8aaefaa5d20e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:38.754053 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f219ac61-6453-430b-bfcb-8aaefaa5d20e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 1.608566 Aug 07 01:12:38.755154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 150/303] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:37 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 1612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:38.798136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3e946816-78af-46fa-a4ce-3a6add6f34ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:38.798973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3e946816-78af-46fa-a4ce-3a6add6f34ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:38.831718 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3e946816-78af-46fa-a4ce-3a6add6f34ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:38.832159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3e946816-78af-46fa-a4ce-3a6add6f34ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:38.837616 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3e946816-78af-46fa-a4ce-3a6add6f34ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:38.839079 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3e946816-78af-46fa-a4ce-3a6add6f34ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:38.840066 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3e946816-78af-46fa-a4ce-3a6add6f34ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:38.841384 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3e946816-78af-46fa-a4ce-3a6add6f34ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:39.142178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-38001036-a6d6-4cd9-8b24-9f47c7bfde90 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:39.205762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-38001036-a6d6-4cd9-8b24-9f47c7bfde90 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:39.207792 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-38001036-a6d6-4cd9-8b24-9f47c7bfde90 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:39.229526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-38001036-a6d6-4cd9-8b24-9f47c7bfde90 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1739 microversion: 2.20 time: 1.670942 Aug 07 01:12:39.241598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 154/304] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:37 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1739 bytes in 1675 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:39.277017 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f1dcf9f7-8ec9-4563-8c37-686cccaf9c2d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:39.278102 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f1dcf9f7-8ec9-4563-8c37-686cccaf9c2d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:39.318636 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f1dcf9f7-8ec9-4563-8c37-686cccaf9c2d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:39.319731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f1dcf9f7-8ec9-4563-8c37-686cccaf9c2d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:39.320554 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f1dcf9f7-8ec9-4563-8c37-686cccaf9c2d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:39.321877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f1dcf9f7-8ec9-4563-8c37-686cccaf9c2d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:39.322681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f1dcf9f7-8ec9-4563-8c37-686cccaf9c2d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:39.323447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f1dcf9f7-8ec9-4563-8c37-686cccaf9c2d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:39.750827 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3e946816-78af-46fa-a4ce-3a6add6f34ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:39.793933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3e946816-78af-46fa-a4ce-3a6add6f34ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 1.036850 Aug 07 01:12:39.795023 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 151/305] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:38 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 1039 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:39.824776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4f01cd4a-e6a0-4aa9-8e35-fe6fea0f99a5 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:39.824776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4f01cd4a-e6a0-4aa9-8e35-fe6fea0f99a5 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:39.865909 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4f01cd4a-e6a0-4aa9-8e35-fe6fea0f99a5 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:39.866827 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4f01cd4a-e6a0-4aa9-8e35-fe6fea0f99a5 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:39.867686 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4f01cd4a-e6a0-4aa9-8e35-fe6fea0f99a5 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:39.869134 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4f01cd4a-e6a0-4aa9-8e35-fe6fea0f99a5 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:39.870034 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4f01cd4a-e6a0-4aa9-8e35-fe6fea0f99a5 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:39.870873 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4f01cd4a-e6a0-4aa9-8e35-fe6fea0f99a5 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:39.935488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f1dcf9f7-8ec9-4563-8c37-686cccaf9c2d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:39.962259 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f1dcf9f7-8ec9-4563-8c37-686cccaf9c2d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1724 microversion: 2.1 time: 0.728438 Aug 07 01:12:39.973098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 155/306] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:39 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1724 bytes in 740 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:39.985526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-8da674bc-d95e-4073-b94f-439528634408 req-334ff765-f9e8-4a3d-8d62-b60b096cee1d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3b71d249-75ce-4352-bb93-34e88cc0df93", "name": "network-vif-plugged", "status": "completed", "tag": "9ea29730-37e9-45db-98d4-1ef7d39b89d9"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:40.000300 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-8da674bc-d95e-4073-b94f-439528634408 req-334ff765-f9e8-4a3d-8d62-b60b096cee1d service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:40.000300 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-8da674bc-d95e-4073-b94f-439528634408 req-334ff765-f9e8-4a3d-8d62-b60b096cee1d service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:40.000300 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-8da674bc-d95e-4073-b94f-439528634408 req-334ff765-f9e8-4a3d-8d62-b60b096cee1d service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:40.023682 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-8da674bc-d95e-4073-b94f-439528634408 req-334ff765-f9e8-4a3d-8d62-b60b096cee1d service nova] Creating event network-vif-plugged:9ea29730-37e9-45db-98d4-1ef7d39b89d9 for instance 3b71d249-75ce-4352-bb93-34e88cc0df93 on hv-cinder-maste Aug 07 01:12:40.035448 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-8da674bc-d95e-4073-b94f-439528634408 req-334ff765-f9e8-4a3d-8d62-b60b096cee1d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060903 Aug 07 01:12:40.036713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 156/307] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:12:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:12:40.045573 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e0ce5623-fbbc-4981-93b6-ce6104b4ad62 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:40.048127 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e0ce5623-fbbc-4981-93b6-ce6104b4ad62 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:40.057759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ce5623-fbbc-4981-93b6-ce6104b4ad62 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:40.058925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ce5623-fbbc-4981-93b6-ce6104b4ad62 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:40.060675 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ce5623-fbbc-4981-93b6-ce6104b4ad62 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:40.062042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ce5623-fbbc-4981-93b6-ce6104b4ad62 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:40.063376 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ce5623-fbbc-4981-93b6-ce6104b4ad62 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:40.064290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ce5623-fbbc-4981-93b6-ce6104b4ad62 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:40.252987 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4f01cd4a-e6a0-4aa9-8e35-fe6fea0f99a5 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:40.282215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4f01cd4a-e6a0-4aa9-8e35-fe6fea0f99a5 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1315 microversion: 2.1 time: 0.485197 Aug 07 01:12:40.284270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 152/308] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:39 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1315 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:40.293407 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a785eec1-61bf-40c3-aac0-be3552a767b6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:40.295008 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a785eec1-61bf-40c3-aac0-be3552a767b6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:40.322968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:40.331011 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a785eec1-61bf-40c3-aac0-be3552a767b6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:40.332379 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a785eec1-61bf-40c3-aac0-be3552a767b6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:40.333453 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a785eec1-61bf-40c3-aac0-be3552a767b6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:40.335059 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a785eec1-61bf-40c3-aac0-be3552a767b6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:40.336196 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a785eec1-61bf-40c3-aac0-be3552a767b6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:40.337304 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a785eec1-61bf-40c3-aac0-be3552a767b6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:40.342883 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e0ce5623-fbbc-4981-93b6-ce6104b4ad62 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:40.357241 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e0ce5623-fbbc-4981-93b6-ce6104b4ad62 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.314818 Aug 07 01:12:40.376291 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 157/309] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:40 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:40.634547 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a785eec1-61bf-40c3-aac0-be3552a767b6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:40.647344 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:40.653263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a785eec1-61bf-40c3-aac0-be3552a767b6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:40.656277 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a785eec1-61bf-40c3-aac0-be3552a767b6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:40.661061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a785eec1-61bf-40c3-aac0-be3552a767b6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1739 microversion: 2.20 time: 0.374283 Aug 07 01:12:40.663019 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 153/310] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:40 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1739 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:40.843295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b422ca53-554a-4d6c-b4a2-e1d5d3a907f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:40.844415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b422ca53-554a-4d6c-b4a2-e1d5d3a907f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:40.859388 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b422ca53-554a-4d6c-b4a2-e1d5d3a907f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:40.860110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b422ca53-554a-4d6c-b4a2-e1d5d3a907f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:40.860696 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b422ca53-554a-4d6c-b4a2-e1d5d3a907f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:40.861915 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b422ca53-554a-4d6c-b4a2-e1d5d3a907f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:40.866909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b422ca53-554a-4d6c-b4a2-e1d5d3a907f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:40.867674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b422ca53-554a-4d6c-b4a2-e1d5d3a907f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:41.032796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-99e8e753-7cb8-4299-b22d-e41a097e3493 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:41.038796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-99e8e753-7cb8-4299-b22d-e41a097e3493 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:41.114738 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-99e8e753-7cb8-4299-b22d-e41a097e3493 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:41.116136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-99e8e753-7cb8-4299-b22d-e41a097e3493 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:41.117085 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-99e8e753-7cb8-4299-b22d-e41a097e3493 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:41.119695 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-99e8e753-7cb8-4299-b22d-e41a097e3493 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:41.120652 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-99e8e753-7cb8-4299-b22d-e41a097e3493 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:41.121517 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-99e8e753-7cb8-4299-b22d-e41a097e3493 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:41.372639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b422ca53-554a-4d6c-b4a2-e1d5d3a907f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:41.383500 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b422ca53-554a-4d6c-b4a2-e1d5d3a907f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.544914 Aug 07 01:12:41.389763 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 158/311] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:40 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 552 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:41.399687 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-34619146-d0aa-4277-8792-0144ed9dd18d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:41.403756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-34619146-d0aa-4277-8792-0144ed9dd18d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:41.422796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-34619146-d0aa-4277-8792-0144ed9dd18d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:41.424155 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-34619146-d0aa-4277-8792-0144ed9dd18d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:41.424943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-34619146-d0aa-4277-8792-0144ed9dd18d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:41.426010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-34619146-d0aa-4277-8792-0144ed9dd18d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:41.426770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-34619146-d0aa-4277-8792-0144ed9dd18d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:41.427427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-34619146-d0aa-4277-8792-0144ed9dd18d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:41.711097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-99e8e753-7cb8-4299-b22d-e41a097e3493 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:41.726988 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-99e8e753-7cb8-4299-b22d-e41a097e3493 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1724 microversion: 2.1 time: 0.723351 Aug 07 01:12:41.730956 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 154/312] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:41 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1724 bytes in 729 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:41.745193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d1d582f3-9598-4503-aa37-90e158d17749 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:41.747879 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d1d582f3-9598-4503-aa37-90e158d17749 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:41.766572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1d582f3-9598-4503-aa37-90e158d17749 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:41.767458 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1d582f3-9598-4503-aa37-90e158d17749 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:41.768230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1d582f3-9598-4503-aa37-90e158d17749 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:41.769292 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1d582f3-9598-4503-aa37-90e158d17749 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:41.771654 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1d582f3-9598-4503-aa37-90e158d17749 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:41.772460 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1d582f3-9598-4503-aa37-90e158d17749 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:41.897442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-34619146-d0aa-4277-8792-0144ed9dd18d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:41.910987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-34619146-d0aa-4277-8792-0144ed9dd18d tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1315 microversion: 2.1 time: 0.519465 Aug 07 01:12:41.912257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 159/313] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:41 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1315 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:41.931716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ed6597a2-d0af-4a4b-a896-ac9b1d763e79 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:41.936514 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ed6597a2-d0af-4a4b-a896-ac9b1d763e79 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:41.958895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed6597a2-d0af-4a4b-a896-ac9b1d763e79 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:41.961126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed6597a2-d0af-4a4b-a896-ac9b1d763e79 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:41.961858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed6597a2-d0af-4a4b-a896-ac9b1d763e79 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:41.962910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed6597a2-d0af-4a4b-a896-ac9b1d763e79 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:41.965572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed6597a2-d0af-4a4b-a896-ac9b1d763e79 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:41.966623 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed6597a2-d0af-4a4b-a896-ac9b1d763e79 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:42.150242 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d1d582f3-9598-4503-aa37-90e158d17749 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:42.192845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d1d582f3-9598-4503-aa37-90e158d17749 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.459813 Aug 07 01:12:42.199522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 155/314] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:41 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:42.416565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-036a2bed-db03-4150-9adf-953bce32741b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:42.417410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-036a2bed-db03-4150-9adf-953bce32741b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:42.462746 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-036a2bed-db03-4150-9adf-953bce32741b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:42.466376 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-036a2bed-db03-4150-9adf-953bce32741b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:42.467600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-036a2bed-db03-4150-9adf-953bce32741b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:42.473148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-036a2bed-db03-4150-9adf-953bce32741b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:42.490604 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-036a2bed-db03-4150-9adf-953bce32741b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:42.503489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-036a2bed-db03-4150-9adf-953bce32741b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:42.638766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ed6597a2-d0af-4a4b-a896-ac9b1d763e79 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:42.656593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ed6597a2-d0af-4a4b-a896-ac9b1d763e79 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:42.667130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ed6597a2-d0af-4a4b-a896-ac9b1d763e79 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:42.669702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ed6597a2-d0af-4a4b-a896-ac9b1d763e79 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1739 microversion: 2.20 time: 0.749412 Aug 07 01:12:42.675417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 160/315] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:41 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1739 bytes in 756 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:42.780191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e673c5ab-39f0-450a-b6c0-600f1e2acedb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:42.780979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e673c5ab-39f0-450a-b6c0-600f1e2acedb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:42.800672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e673c5ab-39f0-450a-b6c0-600f1e2acedb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:42.801363 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e673c5ab-39f0-450a-b6c0-600f1e2acedb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:42.801890 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e673c5ab-39f0-450a-b6c0-600f1e2acedb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:42.802949 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e673c5ab-39f0-450a-b6c0-600f1e2acedb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:42.808422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e673c5ab-39f0-450a-b6c0-600f1e2acedb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:42.808964 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e673c5ab-39f0-450a-b6c0-600f1e2acedb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:42.825693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:43.220591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e673c5ab-39f0-450a-b6c0-600f1e2acedb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:43.241425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e673c5ab-39f0-450a-b6c0-600f1e2acedb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1724 microversion: 2.1 time: 0.471099 Aug 07 01:12:43.256369 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 161/316] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:42 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1724 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:43.256618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-036a2bed-db03-4150-9adf-953bce32741b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:43.257123 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ccf748c6-26c7-47c1-b64e-8f7ed9e9698c tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:43.257123 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ccf748c6-26c7-47c1-b64e-8f7ed9e9698c tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:43.271268 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccf748c6-26c7-47c1-b64e-8f7ed9e9698c tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:43.272203 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccf748c6-26c7-47c1-b64e-8f7ed9e9698c tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:43.273021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccf748c6-26c7-47c1-b64e-8f7ed9e9698c tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:43.274305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccf748c6-26c7-47c1-b64e-8f7ed9e9698c tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:43.275159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccf748c6-26c7-47c1-b64e-8f7ed9e9698c tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:43.275908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccf748c6-26c7-47c1-b64e-8f7ed9e9698c tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:43.289452 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-036a2bed-db03-4150-9adf-953bce32741b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.877670 Aug 07 01:12:43.328863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 156/317] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:42 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 918 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:43.339847 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1ff774ff-1771-441a-acb4-e8064e1fc08d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:43.352479 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1ff774ff-1771-441a-acb4-e8064e1fc08d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:43.360520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ff774ff-1771-441a-acb4-e8064e1fc08d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:43.360520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ff774ff-1771-441a-acb4-e8064e1fc08d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:43.360520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ff774ff-1771-441a-acb4-e8064e1fc08d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:43.360520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ff774ff-1771-441a-acb4-e8064e1fc08d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:43.361247 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ff774ff-1771-441a-acb4-e8064e1fc08d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:43.361247 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ff774ff-1771-441a-acb4-e8064e1fc08d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:43.604321 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ccf748c6-26c7-47c1-b64e-8f7ed9e9698c tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:43.614869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ccf748c6-26c7-47c1-b64e-8f7ed9e9698c tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1315 microversion: 2.1 time: 0.369783 Aug 07 01:12:43.632495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 162/318] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:43 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1315 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:43.653553 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1ff774ff-1771-441a-acb4-e8064e1fc08d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:43.663487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1ff774ff-1771-441a-acb4-e8064e1fc08d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.332790 Aug 07 01:12:43.665105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 157/319] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:43 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:43.719119 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9e2212a4-6499-490b-8600-970df660107b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:43.722478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9e2212a4-6499-490b-8600-970df660107b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:43.733966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e2212a4-6499-490b-8600-970df660107b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:43.735119 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e2212a4-6499-490b-8600-970df660107b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:43.736016 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e2212a4-6499-490b-8600-970df660107b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:43.737627 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e2212a4-6499-490b-8600-970df660107b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:43.738424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e2212a4-6499-490b-8600-970df660107b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:43.739354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e2212a4-6499-490b-8600-970df660107b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:44.034229 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9e2212a4-6499-490b-8600-970df660107b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:44.046869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9e2212a4-6499-490b-8600-970df660107b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:44.049669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9e2212a4-6499-490b-8600-970df660107b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:44.051295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9e2212a4-6499-490b-8600-970df660107b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1739 microversion: 2.20 time: 0.339205 Aug 07 01:12:44.052787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 163/320] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:43 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1739 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:44.272406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-03ca1f0a-75ee-4523-8d81-49989f242bfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:44.273477 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-03ca1f0a-75ee-4523-8d81-49989f242bfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:44.341071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-731d7105-b867-4dcf-869c-410806e8c13c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:44.342517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-731d7105-b867-4dcf-869c-410806e8c13c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:44.352850 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-03ca1f0a-75ee-4523-8d81-49989f242bfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:44.356850 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-03ca1f0a-75ee-4523-8d81-49989f242bfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:44.358433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-03ca1f0a-75ee-4523-8d81-49989f242bfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:44.363162 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-03ca1f0a-75ee-4523-8d81-49989f242bfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:44.367034 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-03ca1f0a-75ee-4523-8d81-49989f242bfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:44.368650 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-03ca1f0a-75ee-4523-8d81-49989f242bfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:44.383582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-731d7105-b867-4dcf-869c-410806e8c13c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:44.388546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-731d7105-b867-4dcf-869c-410806e8c13c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:44.389432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-731d7105-b867-4dcf-869c-410806e8c13c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:44.390820 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-731d7105-b867-4dcf-869c-410806e8c13c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:44.391639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-731d7105-b867-4dcf-869c-410806e8c13c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:44.408602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-731d7105-b867-4dcf-869c-410806e8c13c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:44.852108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-03ca1f0a-75ee-4523-8d81-49989f242bfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:44.876844 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-03ca1f0a-75ee-4523-8d81-49989f242bfb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1724 microversion: 2.1 time: 0.612650 Aug 07 01:12:44.879694 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 158/321] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:44 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1724 bytes in 617 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:44.895250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2735fe2b-c997-486b-bc2f-ca5a9183d8f1 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:44.896929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2735fe2b-c997-486b-bc2f-ca5a9183d8f1 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:44.913619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2735fe2b-c997-486b-bc2f-ca5a9183d8f1 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:44.915917 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2735fe2b-c997-486b-bc2f-ca5a9183d8f1 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:44.919800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2735fe2b-c997-486b-bc2f-ca5a9183d8f1 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:44.921086 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2735fe2b-c997-486b-bc2f-ca5a9183d8f1 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:44.924313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2735fe2b-c997-486b-bc2f-ca5a9183d8f1 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:44.928155 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2735fe2b-c997-486b-bc2f-ca5a9183d8f1 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:45.103088 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-731d7105-b867-4dcf-869c-410806e8c13c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:45.125986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-731d7105-b867-4dcf-869c-410806e8c13c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1322 microversion: 2.1 time: 0.792588 Aug 07 01:12:45.127040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 164/322] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:44 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1322 bytes in 795 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:45.160931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6890eb42-97d2-4801-8c43-60c9a6d5dc0a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:45.162662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6890eb42-97d2-4801-8c43-60c9a6d5dc0a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:45.183915 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6890eb42-97d2-4801-8c43-60c9a6d5dc0a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:45.192300 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6890eb42-97d2-4801-8c43-60c9a6d5dc0a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:45.193556 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6890eb42-97d2-4801-8c43-60c9a6d5dc0a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:45.195369 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6890eb42-97d2-4801-8c43-60c9a6d5dc0a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:45.200068 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6890eb42-97d2-4801-8c43-60c9a6d5dc0a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:45.207074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6890eb42-97d2-4801-8c43-60c9a6d5dc0a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:45.386698 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2735fe2b-c997-486b-bc2f-ca5a9183d8f1 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:45.428632 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2735fe2b-c997-486b-bc2f-ca5a9183d8f1 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1315 microversion: 2.1 time: 0.546232 Aug 07 01:12:45.432405 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 159/323] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:44 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1315 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:45.456125 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-02178973-a99b-4405-b07a-dfa13107963b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:45.462038 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-02178973-a99b-4405-b07a-dfa13107963b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:45.492880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02178973-a99b-4405-b07a-dfa13107963b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:45.499836 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02178973-a99b-4405-b07a-dfa13107963b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:45.503059 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02178973-a99b-4405-b07a-dfa13107963b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:45.505061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02178973-a99b-4405-b07a-dfa13107963b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:45.510628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02178973-a99b-4405-b07a-dfa13107963b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:45.512188 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02178973-a99b-4405-b07a-dfa13107963b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:45.723392 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6890eb42-97d2-4801-8c43-60c9a6d5dc0a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:45.757632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6890eb42-97d2-4801-8c43-60c9a6d5dc0a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.613175 Aug 07 01:12:45.765933 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 165/324] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:45 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 625 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:45.917383 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f823735b-690c-4d88-bc39-5747eab2c05d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:45.925533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f823735b-690c-4d88-bc39-5747eab2c05d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:45.961380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f823735b-690c-4d88-bc39-5747eab2c05d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:45.972173 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-02178973-a99b-4405-b07a-dfa13107963b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:45.974919 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f823735b-690c-4d88-bc39-5747eab2c05d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:45.976516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f823735b-690c-4d88-bc39-5747eab2c05d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:45.985404 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f823735b-690c-4d88-bc39-5747eab2c05d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:45.986860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f823735b-690c-4d88-bc39-5747eab2c05d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:45.993108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f823735b-690c-4d88-bc39-5747eab2c05d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:46.006587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-02178973-a99b-4405-b07a-dfa13107963b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:46.019095 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-02178973-a99b-4405-b07a-dfa13107963b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:46.021893 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-02178973-a99b-4405-b07a-dfa13107963b tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1739 microversion: 2.20 time: 0.585545 Aug 07 01:12:46.023031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 160/325] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:45 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1739 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:46.159328 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-dcbcf354-92ad-4176-9abe-b02ff235ea51 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:46.169077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-dcbcf354-92ad-4176-9abe-b02ff235ea51 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:46.201307 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dcbcf354-92ad-4176-9abe-b02ff235ea51 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:46.202232 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dcbcf354-92ad-4176-9abe-b02ff235ea51 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:46.203004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dcbcf354-92ad-4176-9abe-b02ff235ea51 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:46.204283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dcbcf354-92ad-4176-9abe-b02ff235ea51 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:46.205075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dcbcf354-92ad-4176-9abe-b02ff235ea51 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:46.206454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dcbcf354-92ad-4176-9abe-b02ff235ea51 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:46.608144 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dcbcf354-92ad-4176-9abe-b02ff235ea51 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:46.613201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f823735b-690c-4d88-bc39-5747eab2c05d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:46.618769 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-dcbcf354-92ad-4176-9abe-b02ff235ea51 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1516 microversion: 2.1 time: 0.468632 Aug 07 01:12:46.621038 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 161/326] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:46 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1516 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:46.633515 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d9df6a66-c356-4e8e-beb4-418391d259c4 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:46.634321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d9df6a66-c356-4e8e-beb4-418391d259c4 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:46.661629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9df6a66-c356-4e8e-beb4-418391d259c4 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:46.664683 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9df6a66-c356-4e8e-beb4-418391d259c4 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:46.665858 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9df6a66-c356-4e8e-beb4-418391d259c4 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:46.669906 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9df6a66-c356-4e8e-beb4-418391d259c4 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:46.670195 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f823735b-690c-4d88-bc39-5747eab2c05d tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1724 microversion: 2.1 time: 0.762029 Aug 07 01:12:46.670403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9df6a66-c356-4e8e-beb4-418391d259c4 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:46.691036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9df6a66-c356-4e8e-beb4-418391d259c4 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:46.739927 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 166/327] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:45 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1724 bytes in 833 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:46.746496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0bc2f197-a5be-42da-b5c9-972b9a204e27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:46.761878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0bc2f197-a5be-42da-b5c9-972b9a204e27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:46.800115 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0bc2f197-a5be-42da-b5c9-972b9a204e27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:46.932700 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0bc2f197-a5be-42da-b5c9-972b9a204e27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.108s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:46.934154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0bc2f197-a5be-42da-b5c9-972b9a204e27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.026s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:46.935818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0bc2f197-a5be-42da-b5c9-972b9a204e27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:46.936845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0bc2f197-a5be-42da-b5c9-972b9a204e27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:46.937871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0bc2f197-a5be-42da-b5c9-972b9a204e27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:47.208619 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.objects.instance [None req-0bc2f197-a5be-42da-b5c9-972b9a204e27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lazy-loading 'flavor' on Instance uuid 3b71d249-75ce-4352-bb93-34e88cc0df93 {{(pid=117983) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:12:47.292287 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d9df6a66-c356-4e8e-beb4-418391d259c4 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:47.327969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d9df6a66-c356-4e8e-beb4-418391d259c4 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1315 microversion: 2.1 time: 0.705325 Aug 07 01:12:47.330764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 162/328] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:46 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1315 bytes in 709 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:47.350847 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-acde3ae7-d7cd-40cf-a7fd-f744c3716f43 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:47.351763 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-acde3ae7-d7cd-40cf-a7fd-f744c3716f43 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:47.361289 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acde3ae7-d7cd-40cf-a7fd-f744c3716f43 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:47.362627 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acde3ae7-d7cd-40cf-a7fd-f744c3716f43 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:47.363301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acde3ae7-d7cd-40cf-a7fd-f744c3716f43 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:47.364575 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acde3ae7-d7cd-40cf-a7fd-f744c3716f43 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:47.365386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acde3ae7-d7cd-40cf-a7fd-f744c3716f43 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:47.366136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acde3ae7-d7cd-40cf-a7fd-f744c3716f43 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:47.465900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0bc2f197-a5be-42da-b5c9-972b9a204e27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "POST /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93/action" status: 200 len: 32 microversion: 2.1 time: 0.723650 Aug 07 01:12:47.472102 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 167/329] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:12:46 2022] POST /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93/action => generated 32 bytes in 731 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 07 01:12:47.504350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-20a000f3-2a5b-42f4-8522-4d0201817ece tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:47.506115 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-20a000f3-2a5b-42f4-8522-4d0201817ece tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:47.560225 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20a000f3-2a5b-42f4-8522-4d0201817ece tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:47.561819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20a000f3-2a5b-42f4-8522-4d0201817ece tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:47.563744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20a000f3-2a5b-42f4-8522-4d0201817ece tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:47.566000 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20a000f3-2a5b-42f4-8522-4d0201817ece tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:47.567946 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20a000f3-2a5b-42f4-8522-4d0201817ece tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:47.574772 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20a000f3-2a5b-42f4-8522-4d0201817ece tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:47.957857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-20a000f3-2a5b-42f4-8522-4d0201817ece tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:47.971230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-20a000f3-2a5b-42f4-8522-4d0201817ece tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:47.973628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-20a000f3-2a5b-42f4-8522-4d0201817ece tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:47.976045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-20a000f3-2a5b-42f4-8522-4d0201817ece tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1739 microversion: 2.20 time: 0.487247 Aug 07 01:12:47.996739 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 168/330] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:47 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1739 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:48.009863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c83c7585-6269-46f3-a9cb-3c2b3be1cd13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:48.011092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c83c7585-6269-46f3-a9cb-3c2b3be1cd13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:48.020635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:48.024218 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c83c7585-6269-46f3-a9cb-3c2b3be1cd13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:48.026024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c83c7585-6269-46f3-a9cb-3c2b3be1cd13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:48.027463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c83c7585-6269-46f3-a9cb-3c2b3be1cd13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:48.029122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c83c7585-6269-46f3-a9cb-3c2b3be1cd13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:48.030344 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c83c7585-6269-46f3-a9cb-3c2b3be1cd13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:48.031446 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c83c7585-6269-46f3-a9cb-3c2b3be1cd13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:48.081481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-acde3ae7-d7cd-40cf-a7fd-f744c3716f43 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:48.099777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-acde3ae7-d7cd-40cf-a7fd-f744c3716f43 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.761864 Aug 07 01:12:48.116633 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 163/331] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:47 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 785 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:48.130139 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-26b361af-47bc-4662-8117-ee32be284879 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:48.133108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-26b361af-47bc-4662-8117-ee32be284879 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:48.168805 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26b361af-47bc-4662-8117-ee32be284879 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:48.174665 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26b361af-47bc-4662-8117-ee32be284879 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:48.175341 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26b361af-47bc-4662-8117-ee32be284879 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:48.180474 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26b361af-47bc-4662-8117-ee32be284879 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:48.181405 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26b361af-47bc-4662-8117-ee32be284879 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:48.181973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26b361af-47bc-4662-8117-ee32be284879 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:48.570454 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c83c7585-6269-46f3-a9cb-3c2b3be1cd13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:48.615796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c83c7585-6269-46f3-a9cb-3c2b3be1cd13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.608745 Aug 07 01:12:48.634187 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 169/332] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:48 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 628 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:48.656145 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e6e5646b-9915-4501-b1b8-bcf45d68b289 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:48.657607 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e6e5646b-9915-4501-b1b8-bcf45d68b289 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:48.679616 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6e5646b-9915-4501-b1b8-bcf45d68b289 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:48.680821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6e5646b-9915-4501-b1b8-bcf45d68b289 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:48.681725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6e5646b-9915-4501-b1b8-bcf45d68b289 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:48.684434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6e5646b-9915-4501-b1b8-bcf45d68b289 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:48.685555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6e5646b-9915-4501-b1b8-bcf45d68b289 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:48.686483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6e5646b-9915-4501-b1b8-bcf45d68b289 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:48.757069 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-26b361af-47bc-4662-8117-ee32be284879 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:48.787669 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-26b361af-47bc-4662-8117-ee32be284879 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1701 microversion: 2.1 time: 0.669230 Aug 07 01:12:48.790316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 164/333] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:48 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1701 bytes in 673 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:48.811752 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6b104d0a-0c53-43f3-bbab-1faaed3fd5c2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:48.836089 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b104d0a-0c53-43f3-bbab-1faaed3fd5c2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:48.837022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b104d0a-0c53-43f3-bbab-1faaed3fd5c2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:48.837820 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b104d0a-0c53-43f3-bbab-1faaed3fd5c2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:49.046710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e6e5646b-9915-4501-b1b8-bcf45d68b289 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:49.063198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e6e5646b-9915-4501-b1b8-bcf45d68b289 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1315 microversion: 2.1 time: 0.428417 Aug 07 01:12:49.064162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 170/334] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:48 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1315 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:49.071173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-72f1d984-ee6c-4b32-add9-490d5ee66416 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:49.072012 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-72f1d984-ee6c-4b32-add9-490d5ee66416 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:49.079830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72f1d984-ee6c-4b32-add9-490d5ee66416 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:49.081657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72f1d984-ee6c-4b32-add9-490d5ee66416 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:49.082238 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72f1d984-ee6c-4b32-add9-490d5ee66416 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:49.083295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72f1d984-ee6c-4b32-add9-490d5ee66416 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:49.083772 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72f1d984-ee6c-4b32-add9-490d5ee66416 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:49.084217 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72f1d984-ee6c-4b32-add9-490d5ee66416 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:49.202747 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6b104d0a-0c53-43f3-bbab-1faaed3fd5c2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Going to try to start instance {{(pid=117984) start /opt/stack/nova/nova/compute/api.py:2732}} Aug 07 01:12:49.385508 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6b104d0a-0c53-43f3-bbab-1faaed3fd5c2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "POST /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/action" status: 202 len: 0 microversion: 2.1 time: 0.578140 Aug 07 01:12:49.386865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 165/335] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:12:48 2022] POST /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/action => generated 0 bytes in 581 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 07 01:12:49.400745 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e960923a-7248-46fc-a186-4d0297f4d0b8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:49.402329 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e960923a-7248-46fc-a186-4d0297f4d0b8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:49.422599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e960923a-7248-46fc-a186-4d0297f4d0b8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:49.423580 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e960923a-7248-46fc-a186-4d0297f4d0b8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:49.424396 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e960923a-7248-46fc-a186-4d0297f4d0b8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:49.426109 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e960923a-7248-46fc-a186-4d0297f4d0b8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:49.427309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e960923a-7248-46fc-a186-4d0297f4d0b8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:49.431679 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e960923a-7248-46fc-a186-4d0297f4d0b8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:49.581490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-72f1d984-ee6c-4b32-add9-490d5ee66416 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:49.619096 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-72f1d984-ee6c-4b32-add9-490d5ee66416 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:49.621690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-72f1d984-ee6c-4b32-add9-490d5ee66416 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:49.623652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-72f1d984-ee6c-4b32-add9-490d5ee66416 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1739 microversion: 2.20 time: 0.557792 Aug 07 01:12:49.627797 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 171/336] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:49 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1739 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:49.637353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-545dcf1d-3d94-45cf-8ec0-479a1d27f186 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:49.643918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-545dcf1d-3d94-45cf-8ec0-479a1d27f186 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:49.668486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-545dcf1d-3d94-45cf-8ec0-479a1d27f186 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:49.669301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-545dcf1d-3d94-45cf-8ec0-479a1d27f186 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:49.669953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-545dcf1d-3d94-45cf-8ec0-479a1d27f186 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:49.670917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-545dcf1d-3d94-45cf-8ec0-479a1d27f186 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:49.671602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-545dcf1d-3d94-45cf-8ec0-479a1d27f186 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:49.672295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-545dcf1d-3d94-45cf-8ec0-479a1d27f186 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:49.926742 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e960923a-7248-46fc-a186-4d0297f4d0b8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:49.932085 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-545dcf1d-3d94-45cf-8ec0-479a1d27f186 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:49.947007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-545dcf1d-3d94-45cf-8ec0-479a1d27f186 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1710 microversion: 2.1 time: 0.317509 Aug 07 01:12:49.960278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 172/337] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:49 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1710 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:49.960278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-9aaf6b04-84af-4d36-af70-37f544f82a93 req-61663bbc-2fd1-40da-a6e0-45d81ad21d93 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f92846c8-747c-4e27-ba74-21ca0ee12aab", "name": "network-vif-plugged", "status": "completed", "tag": "fab914e3-36b7-45a0-b7bb-c8529feb8d06"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:49.973034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-9aaf6b04-84af-4d36-af70-37f544f82a93 req-61663bbc-2fd1-40da-a6e0-45d81ad21d93 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:49.973034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-9aaf6b04-84af-4d36-af70-37f544f82a93 req-61663bbc-2fd1-40da-a6e0-45d81ad21d93 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:49.973034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-9aaf6b04-84af-4d36-af70-37f544f82a93 req-61663bbc-2fd1-40da-a6e0-45d81ad21d93 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:49.997624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-9aaf6b04-84af-4d36-af70-37f544f82a93 req-61663bbc-2fd1-40da-a6e0-45d81ad21d93 service nova] Creating event network-vif-plugged:fab914e3-36b7-45a0-b7bb-c8529feb8d06 for instance f92846c8-747c-4e27-ba74-21ca0ee12aab on hv-cinder-maste Aug 07 01:12:50.005645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e960923a-7248-46fc-a186-4d0297f4d0b8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.616688 Aug 07 01:12:50.008436 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 166/338] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:49 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:50.009312 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-9aaf6b04-84af-4d36-af70-37f544f82a93 req-61663bbc-2fd1-40da-a6e0-45d81ad21d93 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055374 Aug 07 01:12:50.010484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 173/339] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:12:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:12:50.083742 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f4d73f6e-4a2f-40b8-aee9-f17da7fd02d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:50.085195 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f4d73f6e-4a2f-40b8-aee9-f17da7fd02d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:50.100311 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f976a465-8f86-4685-90c5-9244fc6c2d84 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:50.101355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f4d73f6e-4a2f-40b8-aee9-f17da7fd02d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:50.102352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f4d73f6e-4a2f-40b8-aee9-f17da7fd02d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:50.102639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f976a465-8f86-4685-90c5-9244fc6c2d84 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:50.103524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f4d73f6e-4a2f-40b8-aee9-f17da7fd02d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:50.104989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f4d73f6e-4a2f-40b8-aee9-f17da7fd02d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:50.105836 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f4d73f6e-4a2f-40b8-aee9-f17da7fd02d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:50.106670 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f4d73f6e-4a2f-40b8-aee9-f17da7fd02d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:50.119513 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f976a465-8f86-4685-90c5-9244fc6c2d84 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:50.132994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f976a465-8f86-4685-90c5-9244fc6c2d84 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:50.134654 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f976a465-8f86-4685-90c5-9244fc6c2d84 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:50.136432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f976a465-8f86-4685-90c5-9244fc6c2d84 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:50.140516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f976a465-8f86-4685-90c5-9244fc6c2d84 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:50.141804 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f976a465-8f86-4685-90c5-9244fc6c2d84 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:50.671024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f976a465-8f86-4685-90c5-9244fc6c2d84 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:50.695354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f976a465-8f86-4685-90c5-9244fc6c2d84 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1315 microversion: 2.1 time: 0.597790 Aug 07 01:12:50.700686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 174/340] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:50 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1315 bytes in 606 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:50.710239 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2fe2842a-7563-499f-9f02-4058b56c56e5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:50.715189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2fe2842a-7563-499f-9f02-4058b56c56e5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:50.739461 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f4d73f6e-4a2f-40b8-aee9-f17da7fd02d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:50.740661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fe2842a-7563-499f-9f02-4058b56c56e5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:50.745529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fe2842a-7563-499f-9f02-4058b56c56e5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:50.746451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fe2842a-7563-499f-9f02-4058b56c56e5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:50.748611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fe2842a-7563-499f-9f02-4058b56c56e5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:50.750043 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fe2842a-7563-499f-9f02-4058b56c56e5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:50.753077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fe2842a-7563-499f-9f02-4058b56c56e5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:50.755368 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f4d73f6e-4a2f-40b8-aee9-f17da7fd02d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.743837 Aug 07 01:12:50.756009 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 167/341] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:50 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 747 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:50.991670 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-449b896d-6328-4025-9345-bb38e9dbe5e9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:50.993024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-449b896d-6328-4025-9345-bb38e9dbe5e9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:51.006745 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-449b896d-6328-4025-9345-bb38e9dbe5e9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:51.007901 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-449b896d-6328-4025-9345-bb38e9dbe5e9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:51.010264 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-449b896d-6328-4025-9345-bb38e9dbe5e9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:51.011418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-449b896d-6328-4025-9345-bb38e9dbe5e9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:51.012181 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-449b896d-6328-4025-9345-bb38e9dbe5e9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:51.012870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-449b896d-6328-4025-9345-bb38e9dbe5e9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:51.060159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2fe2842a-7563-499f-9f02-4058b56c56e5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:51.103802 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2fe2842a-7563-499f-9f02-4058b56c56e5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:51.105276 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2fe2842a-7563-499f-9f02-4058b56c56e5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:51.107925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2fe2842a-7563-499f-9f02-4058b56c56e5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1721 microversion: 2.20 time: 0.405256 Aug 07 01:12:51.109814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 175/342] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:50 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1721 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:51.118887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ed7facc8-5f8f-4352-81e9-b69c33c702c3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:51.120667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ed7facc8-5f8f-4352-81e9-b69c33c702c3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:51.133265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed7facc8-5f8f-4352-81e9-b69c33c702c3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:51.134684 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed7facc8-5f8f-4352-81e9-b69c33c702c3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:51.135921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed7facc8-5f8f-4352-81e9-b69c33c702c3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:51.137712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed7facc8-5f8f-4352-81e9-b69c33c702c3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:51.139354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed7facc8-5f8f-4352-81e9-b69c33c702c3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:51.140277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed7facc8-5f8f-4352-81e9-b69c33c702c3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:51.167098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:51.470218 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ed7facc8-5f8f-4352-81e9-b69c33c702c3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:51.492743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ed7facc8-5f8f-4352-81e9-b69c33c702c3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.380652 Aug 07 01:12:51.508371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 176/343] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:51 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:51.508735 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-449b896d-6328-4025-9345-bb38e9dbe5e9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:51.522501 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-449b896d-6328-4025-9345-bb38e9dbe5e9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1710 microversion: 2.1 time: 0.536022 Aug 07 01:12:51.523412 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 168/344] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:50 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1710 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:51.740153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1ca1233d-58f4-41d3-a6b1-b1894e389042 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:51.741547 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1ca1233d-58f4-41d3-a6b1-b1894e389042 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:51.772427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ca1233d-58f4-41d3-a6b1-b1894e389042 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:51.776369 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ca1233d-58f4-41d3-a6b1-b1894e389042 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:51.781265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ca1233d-58f4-41d3-a6b1-b1894e389042 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:51.787883 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ca1233d-58f4-41d3-a6b1-b1894e389042 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:51.795171 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ca1233d-58f4-41d3-a6b1-b1894e389042 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:51.799485 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e6dfbc38-211f-49c0-9280-3ef02cb1c742 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:51.800330 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ca1233d-58f4-41d3-a6b1-b1894e389042 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:51.810285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e6dfbc38-211f-49c0-9280-3ef02cb1c742 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:51.857927 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6dfbc38-211f-49c0-9280-3ef02cb1c742 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:51.859691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6dfbc38-211f-49c0-9280-3ef02cb1c742 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:51.861108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6dfbc38-211f-49c0-9280-3ef02cb1c742 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:51.863202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6dfbc38-211f-49c0-9280-3ef02cb1c742 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:51.868567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6dfbc38-211f-49c0-9280-3ef02cb1c742 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:51.870008 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6dfbc38-211f-49c0-9280-3ef02cb1c742 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:51.959001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:52.148526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1ca1233d-58f4-41d3-a6b1-b1894e389042 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:52.162752 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1ca1233d-58f4-41d3-a6b1-b1894e389042 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1315 microversion: 2.1 time: 0.436434 Aug 07 01:12:52.164353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 177/345] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:51 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1315 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:52.170308 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-a678a480-e05a-489c-9b6f-4d8f0291b412 req-010e6ffb-c958-4958-a3ef-2b88a0899572 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "54a16243-ecd7-4152-a1d3-bf92cc3709cc", "name": "network-vif-plugged", "status": "completed", "tag": "75592632-690e-4dbf-a132-69141d52e063"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:52.184101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a678a480-e05a-489c-9b6f-4d8f0291b412 req-010e6ffb-c958-4958-a3ef-2b88a0899572 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:52.185059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a678a480-e05a-489c-9b6f-4d8f0291b412 req-010e6ffb-c958-4958-a3ef-2b88a0899572 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:52.185815 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a678a480-e05a-489c-9b6f-4d8f0291b412 req-010e6ffb-c958-4958-a3ef-2b88a0899572 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:52.214261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-a678a480-e05a-489c-9b6f-4d8f0291b412 req-010e6ffb-c958-4958-a3ef-2b88a0899572 service nova] Creating event network-vif-plugged:75592632-690e-4dbf-a132-69141d52e063 for instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc on hv-cinder-maste Aug 07 01:12:52.224663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-a678a480-e05a-489c-9b6f-4d8f0291b412 req-010e6ffb-c958-4958-a3ef-2b88a0899572 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057204 Aug 07 01:12:52.225988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 178/346] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:12:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:12:52.233413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f2661d3c-59ce-4938-8e3e-619cbe9c3f6d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:52.234424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f2661d3c-59ce-4938-8e3e-619cbe9c3f6d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:52.242045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2661d3c-59ce-4938-8e3e-619cbe9c3f6d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:52.242986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2661d3c-59ce-4938-8e3e-619cbe9c3f6d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:52.243707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2661d3c-59ce-4938-8e3e-619cbe9c3f6d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:52.244760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2661d3c-59ce-4938-8e3e-619cbe9c3f6d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:52.245619 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2661d3c-59ce-4938-8e3e-619cbe9c3f6d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:52.246399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2661d3c-59ce-4938-8e3e-619cbe9c3f6d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:52.324872 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e6dfbc38-211f-49c0-9280-3ef02cb1c742 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:52.344552 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e6dfbc38-211f-49c0-9280-3ef02cb1c742 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.564721 Aug 07 01:12:52.346714 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 169/347] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:51 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:52.531505 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e4f633f6-daf7-4f3a-937f-5dca1589f1a9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:52.538884 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e4f633f6-daf7-4f3a-937f-5dca1589f1a9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:52.569393 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4f633f6-daf7-4f3a-937f-5dca1589f1a9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:52.570683 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4f633f6-daf7-4f3a-937f-5dca1589f1a9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:52.571593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4f633f6-daf7-4f3a-937f-5dca1589f1a9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:52.577471 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4f633f6-daf7-4f3a-937f-5dca1589f1a9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:52.578521 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4f633f6-daf7-4f3a-937f-5dca1589f1a9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:52.583426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4f633f6-daf7-4f3a-937f-5dca1589f1a9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:52.667057 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f2661d3c-59ce-4938-8e3e-619cbe9c3f6d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:52.681192 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f2661d3c-59ce-4938-8e3e-619cbe9c3f6d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:52.683663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f2661d3c-59ce-4938-8e3e-619cbe9c3f6d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:52.685438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f2661d3c-59ce-4938-8e3e-619cbe9c3f6d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1721 microversion: 2.20 time: 0.457544 Aug 07 01:12:52.701503 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 179/348] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:52 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1721 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:52.711696 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7b459210-f86f-4dd4-9211-7006ed5f56d6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:52.713650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7b459210-f86f-4dd4-9211-7006ed5f56d6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:52.721649 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b459210-f86f-4dd4-9211-7006ed5f56d6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:52.724723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b459210-f86f-4dd4-9211-7006ed5f56d6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:52.725434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b459210-f86f-4dd4-9211-7006ed5f56d6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:52.726512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b459210-f86f-4dd4-9211-7006ed5f56d6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:52.727328 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b459210-f86f-4dd4-9211-7006ed5f56d6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:52.727947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b459210-f86f-4dd4-9211-7006ed5f56d6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:53.102518 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7b459210-f86f-4dd4-9211-7006ed5f56d6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:53.116493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7b459210-f86f-4dd4-9211-7006ed5f56d6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1710 microversion: 2.1 time: 0.413140 Aug 07 01:12:53.118006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 180/349] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:52 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1710 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:53.202608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f4f3f7c5-bd16-417d-b28e-828e7e25f4ec tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:53.205567 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f4f3f7c5-bd16-417d-b28e-828e7e25f4ec tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:53.228086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4f3f7c5-bd16-417d-b28e-828e7e25f4ec tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:53.229714 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4f3f7c5-bd16-417d-b28e-828e7e25f4ec tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:53.230388 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4f3f7c5-bd16-417d-b28e-828e7e25f4ec tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:53.231345 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4f3f7c5-bd16-417d-b28e-828e7e25f4ec tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:53.232011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4f3f7c5-bd16-417d-b28e-828e7e25f4ec tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:53.232661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4f3f7c5-bd16-417d-b28e-828e7e25f4ec tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:53.393408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e4f633f6-daf7-4f3a-937f-5dca1589f1a9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:53.444828 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e4f633f6-daf7-4f3a-937f-5dca1589f1a9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.918208 Aug 07 01:12:53.445958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 170/350] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:52 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 925 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:53.477601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ba08d0c4-22af-4053-ad5f-2d68bf0bd600 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:53.481564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ba08d0c4-22af-4053-ad5f-2d68bf0bd600 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:53.505973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ba08d0c4-22af-4053-ad5f-2d68bf0bd600 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:53.507882 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ba08d0c4-22af-4053-ad5f-2d68bf0bd600 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:53.509873 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ba08d0c4-22af-4053-ad5f-2d68bf0bd600 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:53.512226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ba08d0c4-22af-4053-ad5f-2d68bf0bd600 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:53.514305 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ba08d0c4-22af-4053-ad5f-2d68bf0bd600 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:53.515961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ba08d0c4-22af-4053-ad5f-2d68bf0bd600 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:53.726595 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f4f3f7c5-bd16-417d-b28e-828e7e25f4ec tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:53.750486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f4f3f7c5-bd16-417d-b28e-828e7e25f4ec tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1465 microversion: 2.1 time: 0.552609 Aug 07 01:12:53.752992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 181/351] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:53 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1465 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:53.762304 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f8371ac5-d532-4e38-b1fa-e55a2134fa17 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:53.764110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f8371ac5-d532-4e38-b1fa-e55a2134fa17 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:53.776689 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8371ac5-d532-4e38-b1fa-e55a2134fa17 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:53.788830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8371ac5-d532-4e38-b1fa-e55a2134fa17 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:53.792829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8371ac5-d532-4e38-b1fa-e55a2134fa17 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:53.795876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8371ac5-d532-4e38-b1fa-e55a2134fa17 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:53.796167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8371ac5-d532-4e38-b1fa-e55a2134fa17 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:53.798464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8371ac5-d532-4e38-b1fa-e55a2134fa17 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:54.115382 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f8371ac5-d532-4e38-b1fa-e55a2134fa17 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:54.130607 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f8371ac5-d532-4e38-b1fa-e55a2134fa17 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:54.137398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f8371ac5-d532-4e38-b1fa-e55a2134fa17 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:54.141376 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f8371ac5-d532-4e38-b1fa-e55a2134fa17 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1721 microversion: 2.20 time: 0.386347 Aug 07 01:12:54.143108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 182/352] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:53 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1721 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:54.153993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d9d16db5-5ff1-47da-9781-04117a1a7e64 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:54.159173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d9d16db5-5ff1-47da-9781-04117a1a7e64 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:54.168515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9d16db5-5ff1-47da-9781-04117a1a7e64 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:54.169734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9d16db5-5ff1-47da-9781-04117a1a7e64 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:54.170796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9d16db5-5ff1-47da-9781-04117a1a7e64 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:54.172298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9d16db5-5ff1-47da-9781-04117a1a7e64 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:54.173103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9d16db5-5ff1-47da-9781-04117a1a7e64 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:54.173764 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9d16db5-5ff1-47da-9781-04117a1a7e64 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:54.209774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d9d16db5-5ff1-47da-9781-04117a1a7e64 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Going to try to terminate instance {{(pid=117983) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:12:54.374304 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d9d16db5-5ff1-47da-9781-04117a1a7e64 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 204 len: 0 microversion: 2.1 time: 0.229252 Aug 07 01:12:54.380032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 183/353] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:12:54 2022] DELETE /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 0 bytes in 233 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 07 01:12:54.387154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9d452e5a-2c6e-4a24-929c-339ae788b55b tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:54.387913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9d452e5a-2c6e-4a24-929c-339ae788b55b tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:54.398263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d452e5a-2c6e-4a24-929c-339ae788b55b tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:54.399773 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d452e5a-2c6e-4a24-929c-339ae788b55b tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:54.400506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d452e5a-2c6e-4a24-929c-339ae788b55b tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:54.401280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d452e5a-2c6e-4a24-929c-339ae788b55b tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:54.401757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d452e5a-2c6e-4a24-929c-339ae788b55b tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:54.402194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d452e5a-2c6e-4a24-929c-339ae788b55b tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:54.413837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ba08d0c4-22af-4053-ad5f-2d68bf0bd600 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:54.453392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ba08d0c4-22af-4053-ad5f-2d68bf0bd600 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.989030 Aug 07 01:12:54.454798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 171/354] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:53 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 1008 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:54.489609 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-60bbf8fc-7cee-4e8e-9b5f-7a58114cb89a tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:54.490726 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-60bbf8fc-7cee-4e8e-9b5f-7a58114cb89a tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:54.504011 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60bbf8fc-7cee-4e8e-9b5f-7a58114cb89a tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:54.505568 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60bbf8fc-7cee-4e8e-9b5f-7a58114cb89a tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:54.507083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60bbf8fc-7cee-4e8e-9b5f-7a58114cb89a tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:54.509180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60bbf8fc-7cee-4e8e-9b5f-7a58114cb89a tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:54.510374 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60bbf8fc-7cee-4e8e-9b5f-7a58114cb89a tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:54.511779 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60bbf8fc-7cee-4e8e-9b5f-7a58114cb89a tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:54.664743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9d452e5a-2c6e-4a24-929c-339ae788b55b tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:54.679248 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9d452e5a-2c6e-4a24-929c-339ae788b55b tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1710 microversion: 2.1 time: 0.300434 Aug 07 01:12:54.680335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 184/355] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:54 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1710 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:54.686970 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-3067dd72-b837-4eb6-a578-8f371fbe3c68 req-7f5d6644-9d24-434b-a2c9-1c807af97937 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7f60fb22-d17a-4b7c-9b7d-95543be1907e", "name": "network-vif-unplugged", "status": "completed", "tag": "9ab59ce3-ed0d-4a8e-b76b-adeff559725a"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:12:54.699495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-3067dd72-b837-4eb6-a578-8f371fbe3c68 req-7f5d6644-9d24-434b-a2c9-1c807af97937 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:54.699987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-3067dd72-b837-4eb6-a578-8f371fbe3c68 req-7f5d6644-9d24-434b-a2c9-1c807af97937 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:54.700477 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-3067dd72-b837-4eb6-a578-8f371fbe3c68 req-7f5d6644-9d24-434b-a2c9-1c807af97937 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:54.715620 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-3067dd72-b837-4eb6-a578-8f371fbe3c68 req-7f5d6644-9d24-434b-a2c9-1c807af97937 service nova] Creating event network-vif-unplugged:9ab59ce3-ed0d-4a8e-b76b-adeff559725a for instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e on hv-cinder-maste Aug 07 01:12:54.722978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-3067dd72-b837-4eb6-a578-8f371fbe3c68 req-7f5d6644-9d24-434b-a2c9-1c807af97937 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.041086 Aug 07 01:12:54.724069 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 185/356] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:12:54 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:12:54.732255 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a203986b-8163-449f-a80b-249c1daaf1a0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:54.733105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a203986b-8163-449f-a80b-249c1daaf1a0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:54.743327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a203986b-8163-449f-a80b-249c1daaf1a0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:54.743888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a203986b-8163-449f-a80b-249c1daaf1a0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:54.744386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a203986b-8163-449f-a80b-249c1daaf1a0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:54.745280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a203986b-8163-449f-a80b-249c1daaf1a0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:54.746242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a203986b-8163-449f-a80b-249c1daaf1a0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:54.746705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a203986b-8163-449f-a80b-249c1daaf1a0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:54.794286 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-60bbf8fc-7cee-4e8e-9b5f-7a58114cb89a tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:54.803984 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-60bbf8fc-7cee-4e8e-9b5f-7a58114cb89a tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1471 microversion: 2.1 time: 0.343368 Aug 07 01:12:54.818830 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 172/357] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:54 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1471 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:55.023293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a203986b-8163-449f-a80b-249c1daaf1a0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:55.061660 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a203986b-8163-449f-a80b-249c1daaf1a0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.320296 Aug 07 01:12:55.061660 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 186/358] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:54 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:55.170292 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0ebde554-3e3a-4cab-a6ae-1916999a9be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:55.170912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0ebde554-3e3a-4cab-a6ae-1916999a9be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:55.178592 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ebde554-3e3a-4cab-a6ae-1916999a9be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:55.179122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ebde554-3e3a-4cab-a6ae-1916999a9be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:55.179731 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ebde554-3e3a-4cab-a6ae-1916999a9be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:55.182522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ebde554-3e3a-4cab-a6ae-1916999a9be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:55.183009 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ebde554-3e3a-4cab-a6ae-1916999a9be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:55.183660 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ebde554-3e3a-4cab-a6ae-1916999a9be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:55.373736 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0ebde554-3e3a-4cab-a6ae-1916999a9be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:55.387164 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0ebde554-3e3a-4cab-a6ae-1916999a9be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:55.390634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0ebde554-3e3a-4cab-a6ae-1916999a9be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:55.392952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0ebde554-3e3a-4cab-a6ae-1916999a9be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1721 microversion: 2.20 time: 0.226707 Aug 07 01:12:55.404229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 173/359] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:55 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1721 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:55.478639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-22c82162-22a0-49a5-ac42-08568c316ed5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:55.479451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-22c82162-22a0-49a5-ac42-08568c316ed5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:55.490166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-22c82162-22a0-49a5-ac42-08568c316ed5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:55.490826 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-22c82162-22a0-49a5-ac42-08568c316ed5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:55.491355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-22c82162-22a0-49a5-ac42-08568c316ed5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:55.492179 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-22c82162-22a0-49a5-ac42-08568c316ed5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:55.493881 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-22c82162-22a0-49a5-ac42-08568c316ed5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:55.493881 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-22c82162-22a0-49a5-ac42-08568c316ed5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:55.706588 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c7f4c8b1-071b-40e1-a167-db12db70efd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:55.710493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c7f4c8b1-071b-40e1-a167-db12db70efd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:55.741861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7f4c8b1-071b-40e1-a167-db12db70efd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:55.743136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7f4c8b1-071b-40e1-a167-db12db70efd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:55.744282 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7f4c8b1-071b-40e1-a167-db12db70efd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:55.745963 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7f4c8b1-071b-40e1-a167-db12db70efd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:55.749909 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7f4c8b1-071b-40e1-a167-db12db70efd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:55.752199 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7f4c8b1-071b-40e1-a167-db12db70efd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:55.770416 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-22c82162-22a0-49a5-ac42-08568c316ed5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:55.784563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-22c82162-22a0-49a5-ac42-08568c316ed5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.307274 Aug 07 01:12:55.795908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 187/360] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:55 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:55.827775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-95310069-1182-4f9a-800d-e8370c98accb tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:55.828317 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-95310069-1182-4f9a-800d-e8370c98accb tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:55.836911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95310069-1182-4f9a-800d-e8370c98accb tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:55.837269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95310069-1182-4f9a-800d-e8370c98accb tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:55.837639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95310069-1182-4f9a-800d-e8370c98accb tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:55.838395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95310069-1182-4f9a-800d-e8370c98accb tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:55.838862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95310069-1182-4f9a-800d-e8370c98accb tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:55.839341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95310069-1182-4f9a-800d-e8370c98accb tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:56.014808 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c7f4c8b1-071b-40e1-a167-db12db70efd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:56.026223 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c7f4c8b1-071b-40e1-a167-db12db70efd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1714 microversion: 2.1 time: 0.327575 Aug 07 01:12:56.026992 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 174/361] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:55 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1714 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:56.061090 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-95310069-1182-4f9a-800d-e8370c98accb tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:56.081061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a5bed6c3-d1d7-4aab-b215-0786fb34b97b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:56.081286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-95310069-1182-4f9a-800d-e8370c98accb tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1471 microversion: 2.1 time: 0.256405 Aug 07 01:12:56.082010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 188/362] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:55 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1471 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:56.082451 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a5bed6c3-d1d7-4aab-b215-0786fb34b97b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:56.098982 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5bed6c3-d1d7-4aab-b215-0786fb34b97b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:56.100128 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5bed6c3-d1d7-4aab-b215-0786fb34b97b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:56.100848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5bed6c3-d1d7-4aab-b215-0786fb34b97b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:56.101862 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5bed6c3-d1d7-4aab-b215-0786fb34b97b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:56.102530 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5bed6c3-d1d7-4aab-b215-0786fb34b97b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:56.103321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5bed6c3-d1d7-4aab-b215-0786fb34b97b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:56.318393 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a5bed6c3-d1d7-4aab-b215-0786fb34b97b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:56.331007 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a5bed6c3-d1d7-4aab-b215-0786fb34b97b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1368 microversion: 2.1 time: 0.255958 Aug 07 01:12:56.332107 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 175/363] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:56 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1368 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:56.421158 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0742358f-9de4-462a-b1b7-304c4a1f9922 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:56.422296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0742358f-9de4-462a-b1b7-304c4a1f9922 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:56.432173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0742358f-9de4-462a-b1b7-304c4a1f9922 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:56.433154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0742358f-9de4-462a-b1b7-304c4a1f9922 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:56.433992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0742358f-9de4-462a-b1b7-304c4a1f9922 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:56.435256 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0742358f-9de4-462a-b1b7-304c4a1f9922 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:56.436278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0742358f-9de4-462a-b1b7-304c4a1f9922 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:56.436998 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0742358f-9de4-462a-b1b7-304c4a1f9922 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:56.721993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0742358f-9de4-462a-b1b7-304c4a1f9922 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:56.734892 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0742358f-9de4-462a-b1b7-304c4a1f9922 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:56.737436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0742358f-9de4-462a-b1b7-304c4a1f9922 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:56.739030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0742358f-9de4-462a-b1b7-304c4a1f9922 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1721 microversion: 2.20 time: 0.323679 Aug 07 01:12:56.740812 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 189/364] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:56 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1721 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:56.813763 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4cccd0fc-5d21-4da5-9034-a450e730de7c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:56.815585 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4cccd0fc-5d21-4da5-9034-a450e730de7c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:56.832718 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4cccd0fc-5d21-4da5-9034-a450e730de7c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:56.833695 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4cccd0fc-5d21-4da5-9034-a450e730de7c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:56.834164 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4cccd0fc-5d21-4da5-9034-a450e730de7c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:56.834965 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4cccd0fc-5d21-4da5-9034-a450e730de7c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:56.835439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4cccd0fc-5d21-4da5-9034-a450e730de7c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:56.835871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4cccd0fc-5d21-4da5-9034-a450e730de7c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:57.087862 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4cccd0fc-5d21-4da5-9034-a450e730de7c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:57.106898 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4cccd0fc-5d21-4da5-9034-a450e730de7c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.299244 Aug 07 01:12:57.108753 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 176/365] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:56 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:57.115191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-475d14e0-b16c-48be-8d82-2bd7a0357a5e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:57.119679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-475d14e0-b16c-48be-8d82-2bd7a0357a5e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:57.131664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-475d14e0-b16c-48be-8d82-2bd7a0357a5e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:57.132567 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-475d14e0-b16c-48be-8d82-2bd7a0357a5e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:57.133368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-475d14e0-b16c-48be-8d82-2bd7a0357a5e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:57.134706 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-475d14e0-b16c-48be-8d82-2bd7a0357a5e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:57.135527 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-475d14e0-b16c-48be-8d82-2bd7a0357a5e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:57.136347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-475d14e0-b16c-48be-8d82-2bd7a0357a5e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:57.335276 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-475d14e0-b16c-48be-8d82-2bd7a0357a5e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:57.344170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-475d14e0-b16c-48be-8d82-2bd7a0357a5e tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1471 microversion: 2.1 time: 0.239411 Aug 07 01:12:57.348289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 190/366] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:57 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1471 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:57.370067 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5c8579af-865a-40d7-85fa-5761d33b21be tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:57.370774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5c8579af-865a-40d7-85fa-5761d33b21be tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:57.379331 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c8579af-865a-40d7-85fa-5761d33b21be tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:57.380735 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c8579af-865a-40d7-85fa-5761d33b21be tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:57.381557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c8579af-865a-40d7-85fa-5761d33b21be tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:57.382908 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c8579af-865a-40d7-85fa-5761d33b21be tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:57.383956 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c8579af-865a-40d7-85fa-5761d33b21be tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:57.384699 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c8579af-865a-40d7-85fa-5761d33b21be tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:57.584827 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5c8579af-865a-40d7-85fa-5761d33b21be tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:57.593884 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5c8579af-865a-40d7-85fa-5761d33b21be tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1559 microversion: 2.1 time: 0.228694 Aug 07 01:12:57.594489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 177/367] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:57 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1559 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:12:57.620467 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0f8cfd33-2212-4de0-89e5-7863d45fc390 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:57.621322 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0f8cfd33-2212-4de0-89e5-7863d45fc390 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:57.631510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f8cfd33-2212-4de0-89e5-7863d45fc390 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:57.632267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f8cfd33-2212-4de0-89e5-7863d45fc390 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:57.633079 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f8cfd33-2212-4de0-89e5-7863d45fc390 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:57.634018 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f8cfd33-2212-4de0-89e5-7863d45fc390 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:57.634936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f8cfd33-2212-4de0-89e5-7863d45fc390 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:57.635724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f8cfd33-2212-4de0-89e5-7863d45fc390 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:57.787563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0f8cfd33-2212-4de0-89e5-7863d45fc390 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.171752 Aug 07 01:12:57.788723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 191/368] 10.222.0.26 () {58 vars in 1275 bytes} [Sun Aug 7 01:12:57 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc/os-interface => generated 285 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:12:57.797963 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-dd75df8b-ed77-40e0-8567-eaa2df26908c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:57.798632 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-dd75df8b-ed77-40e0-8567-eaa2df26908c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:57.834042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dd75df8b-ed77-40e0-8567-eaa2df26908c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:57.836284 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dd75df8b-ed77-40e0-8567-eaa2df26908c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:57.836284 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dd75df8b-ed77-40e0-8567-eaa2df26908c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:57.838625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dd75df8b-ed77-40e0-8567-eaa2df26908c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:57.839052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dd75df8b-ed77-40e0-8567-eaa2df26908c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:57.840899 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dd75df8b-ed77-40e0-8567-eaa2df26908c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:58.142602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e066b9ee-6f47-484f-8fa7-5823d4a869cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:58.144174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e066b9ee-6f47-484f-8fa7-5823d4a869cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:58.153410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e066b9ee-6f47-484f-8fa7-5823d4a869cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:58.154165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e066b9ee-6f47-484f-8fa7-5823d4a869cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:58.154936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e066b9ee-6f47-484f-8fa7-5823d4a869cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:58.155841 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e066b9ee-6f47-484f-8fa7-5823d4a869cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:58.156824 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e066b9ee-6f47-484f-8fa7-5823d4a869cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:58.157298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e066b9ee-6f47-484f-8fa7-5823d4a869cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:58.219674 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dd75df8b-ed77-40e0-8567-eaa2df26908c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:58.257213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dd75df8b-ed77-40e0-8567-eaa2df26908c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:58.271087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dd75df8b-ed77-40e0-8567-eaa2df26908c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:58.275184 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-dd75df8b-ed77-40e0-8567-eaa2df26908c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1721 microversion: 2.20 time: 0.499159 Aug 07 01:12:58.279523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 178/369] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:57 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1721 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:58.371946 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-59b0ba50-fc97-4abe-b0f9-81c95c991a46 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:58.381998 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-59b0ba50-fc97-4abe-b0f9-81c95c991a46 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:58.409460 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59b0ba50-fc97-4abe-b0f9-81c95c991a46 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:58.411616 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59b0ba50-fc97-4abe-b0f9-81c95c991a46 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:58.412763 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59b0ba50-fc97-4abe-b0f9-81c95c991a46 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:58.414648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59b0ba50-fc97-4abe-b0f9-81c95c991a46 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:58.415800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59b0ba50-fc97-4abe-b0f9-81c95c991a46 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:58.416943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59b0ba50-fc97-4abe-b0f9-81c95c991a46 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:58.444965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e066b9ee-6f47-484f-8fa7-5823d4a869cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:58.471512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e066b9ee-6f47-484f-8fa7-5823d4a869cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.334344 Aug 07 01:12:58.474522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 192/370] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:58 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:58.887829 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-59b0ba50-fc97-4abe-b0f9-81c95c991a46 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:58.900123 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-59b0ba50-fc97-4abe-b0f9-81c95c991a46 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1471 microversion: 2.1 time: 0.535457 Aug 07 01:12:58.923628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 179/371] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:58 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1471 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:12:59.308413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-04b1c2f6-bb45-4e7e-a036-01221db5c35f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:59.309979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-04b1c2f6-bb45-4e7e-a036-01221db5c35f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:59.325591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04b1c2f6-bb45-4e7e-a036-01221db5c35f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:59.327502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04b1c2f6-bb45-4e7e-a036-01221db5c35f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:59.328226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04b1c2f6-bb45-4e7e-a036-01221db5c35f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:59.329195 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04b1c2f6-bb45-4e7e-a036-01221db5c35f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:59.330537 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04b1c2f6-bb45-4e7e-a036-01221db5c35f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:59.331517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04b1c2f6-bb45-4e7e-a036-01221db5c35f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:59.338748 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:12:59.514698 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ae9eb36f-7e79-48c9-9a9d-ba0cb9362b1b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:59.515440 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ae9eb36f-7e79-48c9-9a9d-ba0cb9362b1b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:12:59.558268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae9eb36f-7e79-48c9-9a9d-ba0cb9362b1b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:59.559222 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae9eb36f-7e79-48c9-9a9d-ba0cb9362b1b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:59.560226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae9eb36f-7e79-48c9-9a9d-ba0cb9362b1b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:59.561821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae9eb36f-7e79-48c9-9a9d-ba0cb9362b1b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:12:59.562659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae9eb36f-7e79-48c9-9a9d-ba0cb9362b1b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:12:59.563498 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae9eb36f-7e79-48c9-9a9d-ba0cb9362b1b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:12:59.645045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-04b1c2f6-bb45-4e7e-a036-01221db5c35f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:59.665234 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-04b1c2f6-bb45-4e7e-a036-01221db5c35f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:59.666889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-04b1c2f6-bb45-4e7e-a036-01221db5c35f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:12:59.668636 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-04b1c2f6-bb45-4e7e-a036-01221db5c35f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1721 microversion: 2.20 time: 0.368257 Aug 07 01:12:59.670247 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 193/372] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:12:59 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1721 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:12:59.971903 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d06645cc-674e-4628-b5be-a4e3eb060ed3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:12:59.986758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d06645cc-674e-4628-b5be-a4e3eb060ed3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:00.045520 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d06645cc-674e-4628-b5be-a4e3eb060ed3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:00.053615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d06645cc-674e-4628-b5be-a4e3eb060ed3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:00.054672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d06645cc-674e-4628-b5be-a4e3eb060ed3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:00.063109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d06645cc-674e-4628-b5be-a4e3eb060ed3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:00.071716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d06645cc-674e-4628-b5be-a4e3eb060ed3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:00.074728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d06645cc-674e-4628-b5be-a4e3eb060ed3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:00.257906 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ae9eb36f-7e79-48c9-9a9d-ba0cb9362b1b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:00.282845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ae9eb36f-7e79-48c9-9a9d-ba0cb9362b1b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.777174 Aug 07 01:13:00.297663 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 180/373] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:59 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 793 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:00.324445 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-a4457a1d-4bea-4e85-bc3f-fc953a01e5b1 req-21edd5cb-4c55-4109-92ff-e63581c134b8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "54a16243-ecd7-4152-a1d3-bf92cc3709cc", "tag": "75592632-690e-4dbf-a132-69141d52e063"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:00.353944 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-a4457a1d-4bea-4e85-bc3f-fc953a01e5b1 req-21edd5cb-4c55-4109-92ff-e63581c134b8 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:00.356925 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-a4457a1d-4bea-4e85-bc3f-fc953a01e5b1 req-21edd5cb-4c55-4109-92ff-e63581c134b8 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:00.357767 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-a4457a1d-4bea-4e85-bc3f-fc953a01e5b1 req-21edd5cb-4c55-4109-92ff-e63581c134b8 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:00.421774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-a4457a1d-4bea-4e85-bc3f-fc953a01e5b1 req-21edd5cb-4c55-4109-92ff-e63581c134b8 service nova] Creating event network-changed:75592632-690e-4dbf-a132-69141d52e063 for instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc on hv-cinder-maste Aug 07 01:13:00.438869 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-a4457a1d-4bea-4e85-bc3f-fc953a01e5b1 req-21edd5cb-4c55-4109-92ff-e63581c134b8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.139714 Aug 07 01:13:00.445363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 181/374] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:13:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:13:00.709432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-22649ac5-82ea-411c-9ab2-089545417b67 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:00.710259 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-22649ac5-82ea-411c-9ab2-089545417b67 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:00.730498 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-22649ac5-82ea-411c-9ab2-089545417b67 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:00.734348 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-22649ac5-82ea-411c-9ab2-089545417b67 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:00.734920 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-22649ac5-82ea-411c-9ab2-089545417b67 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:00.735697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-22649ac5-82ea-411c-9ab2-089545417b67 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:00.736719 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-22649ac5-82ea-411c-9ab2-089545417b67 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:00.741804 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-22649ac5-82ea-411c-9ab2-089545417b67 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:00.864024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d06645cc-674e-4628-b5be-a4e3eb060ed3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:00.890972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d06645cc-674e-4628-b5be-a4e3eb060ed3 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1471 microversion: 2.1 time: 0.928531 Aug 07 01:13:00.892032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 194/375] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:12:59 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1471 bytes in 931 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:01.326910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-acb2dfa5-c406-45c8-84ff-7896700d9bc4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:01.327921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-acb2dfa5-c406-45c8-84ff-7896700d9bc4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:01.351795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acb2dfa5-c406-45c8-84ff-7896700d9bc4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:01.353660 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acb2dfa5-c406-45c8-84ff-7896700d9bc4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:01.354934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acb2dfa5-c406-45c8-84ff-7896700d9bc4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:01.356957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acb2dfa5-c406-45c8-84ff-7896700d9bc4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:01.359663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acb2dfa5-c406-45c8-84ff-7896700d9bc4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:01.362274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acb2dfa5-c406-45c8-84ff-7896700d9bc4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:01.831948 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-22649ac5-82ea-411c-9ab2-089545417b67 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:01.921246 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-22649ac5-82ea-411c-9ab2-089545417b67 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:01.929200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-22649ac5-82ea-411c-9ab2-089545417b67 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:01.953474 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-22649ac5-82ea-411c-9ab2-089545417b67 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1724 microversion: 2.20 time: 1.247555 Aug 07 01:13:01.958558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 182/376] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:00 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1724 bytes in 1254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:13:01.980943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bf86fa31-8554-4f6b-a2e4-362ae0eb3427 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:01.982990 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bf86fa31-8554-4f6b-a2e4-362ae0eb3427 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:02.009772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bf86fa31-8554-4f6b-a2e4-362ae0eb3427 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:02.011139 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bf86fa31-8554-4f6b-a2e4-362ae0eb3427 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:02.012028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bf86fa31-8554-4f6b-a2e4-362ae0eb3427 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:02.018839 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bf86fa31-8554-4f6b-a2e4-362ae0eb3427 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:02.020162 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bf86fa31-8554-4f6b-a2e4-362ae0eb3427 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:02.023907 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bf86fa31-8554-4f6b-a2e4-362ae0eb3427 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:02.360643 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-acb2dfa5-c406-45c8-84ff-7896700d9bc4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:02.384443 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-acb2dfa5-c406-45c8-84ff-7896700d9bc4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 1.072774 Aug 07 01:13:02.414064 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 195/377] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:01 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 1104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:02.434913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-172611ce-a3ef-472a-9968-933a0ff43a69 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9bffc979-92b9-42b4-a2f9-1d3f5dd0e21c"}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:02.440315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-172611ce-a3ef-472a-9968-933a0ff43a69 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:02.445566 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bf86fa31-8554-4f6b-a2e4-362ae0eb3427 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab3ed0f7df742618c77815ef1ddb960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4203fc81bbd14439bb6ce329a5ccbc79', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:02.453289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-172611ce-a3ef-472a-9968-933a0ff43a69 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:02.454617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-172611ce-a3ef-472a-9968-933a0ff43a69 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:02.455679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-172611ce-a3ef-472a-9968-933a0ff43a69 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:02.457606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-172611ce-a3ef-472a-9968-933a0ff43a69 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:02.458800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-172611ce-a3ef-472a-9968-933a0ff43a69 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:02.459820 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-172611ce-a3ef-472a-9968-933a0ff43a69 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:02.503470 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bf86fa31-8554-4f6b-a2e4-362ae0eb3427 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 200 len: 1429 microversion: 2.1 time: 0.539628 Aug 07 01:13:02.523425 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 183/378] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:01 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 1429 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:02.565626 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-de46fd62-13f4-4ef5-87b7-fa6e870fea9e req-a2fdaac9-6033-4521-aa24-2fef1bc318a5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f92846c8-747c-4e27-ba74-21ca0ee12aab", "name": "network-vif-deleted", "tag": "fab914e3-36b7-45a0-b7bb-c8529feb8d06"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:02.622859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-de46fd62-13f4-4ef5-87b7-fa6e870fea9e req-a2fdaac9-6033-4521-aa24-2fef1bc318a5 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:02.624696 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-de46fd62-13f4-4ef5-87b7-fa6e870fea9e req-a2fdaac9-6033-4521-aa24-2fef1bc318a5 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:02.628796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-de46fd62-13f4-4ef5-87b7-fa6e870fea9e req-a2fdaac9-6033-4521-aa24-2fef1bc318a5 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:02.670203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-de46fd62-13f4-4ef5-87b7-fa6e870fea9e req-a2fdaac9-6033-4521-aa24-2fef1bc318a5 service nova] Creating event network-vif-deleted:fab914e3-36b7-45a0-b7bb-c8529feb8d06 for instance f92846c8-747c-4e27-ba74-21ca0ee12aab on hv-cinder-maste Aug 07 01:13:02.697113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-de46fd62-13f4-4ef5-87b7-fa6e870fea9e req-a2fdaac9-6033-4521-aa24-2fef1bc318a5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.160290 Aug 07 01:13:02.698788 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 184/379] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:13:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:13:03.286617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-172611ce-a3ef-472a-9968-933a0ff43a69 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "POST /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.870600 Aug 07 01:13:03.287259 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 196/380] 10.222.0.26 () {62 vars in 1364 bytes} [Sun Aug 7 01:13:02 2022] POST /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/os-volume_attachments => generated 188 bytes in 872 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:13:03.444774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d7fbdab9-c8bb-4837-8b85-7b14659f3d80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:03.450970 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d7fbdab9-c8bb-4837-8b85-7b14659f3d80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:03.478738 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7fbdab9-c8bb-4837-8b85-7b14659f3d80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:03.479644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7fbdab9-c8bb-4837-8b85-7b14659f3d80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:03.493313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7fbdab9-c8bb-4837-8b85-7b14659f3d80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:03.496871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7fbdab9-c8bb-4837-8b85-7b14659f3d80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:03.497875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7fbdab9-c8bb-4837-8b85-7b14659f3d80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:03.500781 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7fbdab9-c8bb-4837-8b85-7b14659f3d80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:03.516018 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f3bf131e-3f04-43a2-a371-6101765f9244 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:03.516825 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f3bf131e-3f04-43a2-a371-6101765f9244 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:03.536049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f3bf131e-3f04-43a2-a371-6101765f9244 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:03.538693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f3bf131e-3f04-43a2-a371-6101765f9244 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:03.539769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f3bf131e-3f04-43a2-a371-6101765f9244 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:03.541263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f3bf131e-3f04-43a2-a371-6101765f9244 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:03.542120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f3bf131e-3f04-43a2-a371-6101765f9244 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:03.542749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f3bf131e-3f04-43a2-a371-6101765f9244 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:03.632787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f3bf131e-3f04-43a2-a371-6101765f9244 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=117983) unshelve /opt/stack/nova/nova/compute/api.py:4531}} Aug 07 01:13:03.772800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f3bf131e-3f04-43a2-a371-6101765f9244 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "POST /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/action" status: 202 len: 0 microversion: 2.20 time: 0.261539 Aug 07 01:13:03.776197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 197/381] 10.222.0.26 () {62 vars in 1319 bytes} [Sun Aug 7 01:13:03 2022] POST /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/action => generated 0 bytes in 266 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 07 01:13:03.782587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-60fe6ab6-1e3c-4612-980d-15ccbdb4c1a8 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:03.784559 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-60fe6ab6-1e3c-4612-980d-15ccbdb4c1a8 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] [instance: f92846c8-747c-4e27-ba74-21ca0ee12aab] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:03.811603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60fe6ab6-1e3c-4612-980d-15ccbdb4c1a8 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:03.812769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60fe6ab6-1e3c-4612-980d-15ccbdb4c1a8 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:03.813525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60fe6ab6-1e3c-4612-980d-15ccbdb4c1a8 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:03.814844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60fe6ab6-1e3c-4612-980d-15ccbdb4c1a8 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:03.815713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60fe6ab6-1e3c-4612-980d-15ccbdb4c1a8 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:03.816497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60fe6ab6-1e3c-4612-980d-15ccbdb4c1a8 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:03.834693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-60fe6ab6-1e3c-4612-980d-15ccbdb4c1a8 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] HTTP exception thrown: Instance f92846c8-747c-4e27-ba74-21ca0ee12aab could not be found. Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-60fe6ab6-1e3c-4612-980d-15ccbdb4c1a8 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Request method failure captured: Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab HTTP/1.1 Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xFqH9fCmB6KRvh6DkmfLCCU60Td9ZRUO74fyb6598P4Xht5WfS5es8st_B9MIa6IyYYOOmIcAHj2zcpGqgiiDV--mxAdZbJr_YhXiWQFkBmTIIpwV2vFmGtc1fHLHjD2mS5w0HuI0rm-Id_R_0pIzh6Q7kAtIDZ_YMjGv0vgmi1lI0 Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 4203fc81bbd14439bb6ce329a5ccbc79 Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-VolumesNegativeTest-1782156448 Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:13:03.837282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:13:03.838688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4203fc81bbd14439bb6ce329a5ccbc79"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/4203fc81bbd14439bb6ce329a5ccbc79"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_4203fc81bbd14439bb6ce329a5ccbc79"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4203fc81bbd14439bb6ce329a5ccbc79"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:13:03.838688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-VolumesNegativeTest-1782156448 Aug 07 01:13:03.838688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 4203fc81bbd14439bb6ce329a5ccbc79 Aug 07 01:13:03.838688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-VolumesNegativeTest-1782156448 Aug 07 01:13:03.838688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-VolumesNegativeTest-1782156448-project Aug 07 01:13:03.838688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:13:03.838688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:13:03.838688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 0ab3ed0f7df742618c77815ef1ddb960 Aug 07 01:13:03.838688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-VolumesNegativeTest-1782156448-project Aug 07 01:13:03.838688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:13:03.838688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance f92846c8-747c-4e27-ba74-21ca0ee12aab could not be found. Aug 07 01:13:03.838688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': 'f92846c8-747c-4e27-ba74-21ca0ee12aab'} Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f92846c8-747c-4e27-ba74-21ca0ee12aab could not be found. Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:13:03.839624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f92846c8-747c-4e27-ba74-21ca0ee12aab could not be found. Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f92846c8-747c-4e27-ba74-21ca0ee12aab could not be found. Aug 07 01:13:03.841098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:13:03.842203 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-60fe6ab6-1e3c-4612-980d-15ccbdb4c1a8 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] Returning 404 to user: Instance f92846c8-747c-4e27-ba74-21ca0ee12aab could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:13:03.842203 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-60fe6ab6-1e3c-4612-980d-15ccbdb4c1a8 tempest-VolumesNegativeTest-1782156448 tempest-VolumesNegativeTest-1782156448-project] 10.222.0.26 "GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab" status: 404 len: 111 microversion: 2.1 time: 0.061841 Aug 07 01:13:03.844719 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 198/382] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:03 2022] GET /compute/v2.1/servers/f92846c8-747c-4e27-ba74-21ca0ee12aab => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:13:03.851799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c70ee144-809f-4463-835b-fe86104d58c2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:03.852823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c70ee144-809f-4463-835b-fe86104d58c2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:03.860728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c70ee144-809f-4463-835b-fe86104d58c2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:03.861543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c70ee144-809f-4463-835b-fe86104d58c2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:03.862328 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c70ee144-809f-4463-835b-fe86104d58c2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:03.863451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c70ee144-809f-4463-835b-fe86104d58c2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:03.864134 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c70ee144-809f-4463-835b-fe86104d58c2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:03.864977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c70ee144-809f-4463-835b-fe86104d58c2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:03.999627 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d7fbdab9-c8bb-4837-8b85-7b14659f3d80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:04.024116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d7fbdab9-c8bb-4837-8b85-7b14659f3d80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.592085 Aug 07 01:13:04.025170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 185/383] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:03 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:04.087226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c70ee144-809f-4463-835b-fe86104d58c2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:04.098357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c70ee144-809f-4463-835b-fe86104d58c2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:04.100497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c70ee144-809f-4463-835b-fe86104d58c2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:04.102324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c70ee144-809f-4463-835b-fe86104d58c2 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1750 microversion: 2.20 time: 0.255397 Aug 07 01:13:04.103120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 199/384] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:03 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1750 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:05.060665 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6d13cc48-7ed5-46ab-a3cf-51faa1dc6fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:05.062328 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6d13cc48-7ed5-46ab-a3cf-51faa1dc6fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:05.074921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6d13cc48-7ed5-46ab-a3cf-51faa1dc6fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:05.075831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6d13cc48-7ed5-46ab-a3cf-51faa1dc6fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:05.076563 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6d13cc48-7ed5-46ab-a3cf-51faa1dc6fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:05.078417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6d13cc48-7ed5-46ab-a3cf-51faa1dc6fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:05.079509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6d13cc48-7ed5-46ab-a3cf-51faa1dc6fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:05.080525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6d13cc48-7ed5-46ab-a3cf-51faa1dc6fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:05.131327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-85355797-f329-4c62-a7e6-1f533c96e3fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:05.133254 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-85355797-f329-4c62-a7e6-1f533c96e3fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:05.144976 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-85355797-f329-4c62-a7e6-1f533c96e3fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:05.146305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-85355797-f329-4c62-a7e6-1f533c96e3fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:05.147433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-85355797-f329-4c62-a7e6-1f533c96e3fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:05.149113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-85355797-f329-4c62-a7e6-1f533c96e3fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:05.150253 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-85355797-f329-4c62-a7e6-1f533c96e3fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:05.151464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-85355797-f329-4c62-a7e6-1f533c96e3fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:05.413202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6d13cc48-7ed5-46ab-a3cf-51faa1dc6fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:05.427701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-85355797-f329-4c62-a7e6-1f533c96e3fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:05.442169 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6d13cc48-7ed5-46ab-a3cf-51faa1dc6fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.387627 Aug 07 01:13:05.443204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 186/385] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:05 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:05.462974 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-85355797-f329-4c62-a7e6-1f533c96e3fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:05.473760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-85355797-f329-4c62-a7e6-1f533c96e3fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:05.476517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-85355797-f329-4c62-a7e6-1f533c96e3fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1750 microversion: 2.20 time: 0.351703 Aug 07 01:13:05.478196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 200/386] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:05 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1750 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:06.482170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b17fa1b3-f50a-4235-b3b0-4d6a3aabcc8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:06.483838 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b17fa1b3-f50a-4235-b3b0-4d6a3aabcc8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:06.493526 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b17fa1b3-f50a-4235-b3b0-4d6a3aabcc8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:06.494594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b17fa1b3-f50a-4235-b3b0-4d6a3aabcc8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:06.495544 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b17fa1b3-f50a-4235-b3b0-4d6a3aabcc8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:06.496817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b17fa1b3-f50a-4235-b3b0-4d6a3aabcc8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:06.497835 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b17fa1b3-f50a-4235-b3b0-4d6a3aabcc8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:06.498696 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b17fa1b3-f50a-4235-b3b0-4d6a3aabcc8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:06.530916 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-954bbaf1-3e7d-4102-94ed-18e863192dc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:06.532655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-954bbaf1-3e7d-4102-94ed-18e863192dc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:06.541028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:13:06.545664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-954bbaf1-3e7d-4102-94ed-18e863192dc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:06.546671 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-954bbaf1-3e7d-4102-94ed-18e863192dc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:06.547659 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-954bbaf1-3e7d-4102-94ed-18e863192dc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:06.549733 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-954bbaf1-3e7d-4102-94ed-18e863192dc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:06.550754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-954bbaf1-3e7d-4102-94ed-18e863192dc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:06.551679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-954bbaf1-3e7d-4102-94ed-18e863192dc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:06.715234 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b17fa1b3-f50a-4235-b3b0-4d6a3aabcc8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:06.724503 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b17fa1b3-f50a-4235-b3b0-4d6a3aabcc8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.248478 Aug 07 01:13:06.727219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 187/387] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:06 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:06.775395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-954bbaf1-3e7d-4102-94ed-18e863192dc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:06.792863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-954bbaf1-3e7d-4102-94ed-18e863192dc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:06.796005 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-954bbaf1-3e7d-4102-94ed-18e863192dc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:06.805033 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-954bbaf1-3e7d-4102-94ed-18e863192dc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1750 microversion: 2.20 time: 0.280619 Aug 07 01:13:06.806486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 201/388] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:06 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1750 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:13:07.755217 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-617b93dc-6392-4b95-88fe-cf4f28dde976 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:07.756224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-617b93dc-6392-4b95-88fe-cf4f28dde976 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:07.765663 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-617b93dc-6392-4b95-88fe-cf4f28dde976 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:07.766297 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-617b93dc-6392-4b95-88fe-cf4f28dde976 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:07.766945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-617b93dc-6392-4b95-88fe-cf4f28dde976 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:07.767833 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-617b93dc-6392-4b95-88fe-cf4f28dde976 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:07.768652 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-617b93dc-6392-4b95-88fe-cf4f28dde976 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:07.769204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-617b93dc-6392-4b95-88fe-cf4f28dde976 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:07.844280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d9ce67a3-2fd6-4dc6-8f4e-1a80eda5183d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:07.845164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d9ce67a3-2fd6-4dc6-8f4e-1a80eda5183d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:07.883126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9ce67a3-2fd6-4dc6-8f4e-1a80eda5183d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:07.884066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9ce67a3-2fd6-4dc6-8f4e-1a80eda5183d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:07.884632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9ce67a3-2fd6-4dc6-8f4e-1a80eda5183d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:07.885964 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9ce67a3-2fd6-4dc6-8f4e-1a80eda5183d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:07.886616 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9ce67a3-2fd6-4dc6-8f4e-1a80eda5183d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:07.887435 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9ce67a3-2fd6-4dc6-8f4e-1a80eda5183d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:07.995176 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-617b93dc-6392-4b95-88fe-cf4f28dde976 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:08.006676 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-617b93dc-6392-4b95-88fe-cf4f28dde976 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.258935 Aug 07 01:13:08.007645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 188/389] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:07 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:08.153314 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d9ce67a3-2fd6-4dc6-8f4e-1a80eda5183d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:08.167450 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d9ce67a3-2fd6-4dc6-8f4e-1a80eda5183d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:08.169959 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d9ce67a3-2fd6-4dc6-8f4e-1a80eda5183d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:08.171620 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d9ce67a3-2fd6-4dc6-8f4e-1a80eda5183d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1750 microversion: 2.20 time: 0.331853 Aug 07 01:13:08.173120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 202/390] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:07 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1750 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:09.029630 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8116253b-35e9-4a02-a639-e9f8771c03e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:09.031140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8116253b-35e9-4a02-a639-e9f8771c03e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:09.038172 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8116253b-35e9-4a02-a639-e9f8771c03e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:09.038970 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8116253b-35e9-4a02-a639-e9f8771c03e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:09.039641 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8116253b-35e9-4a02-a639-e9f8771c03e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:09.040417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8116253b-35e9-4a02-a639-e9f8771c03e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:09.040912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8116253b-35e9-4a02-a639-e9f8771c03e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:09.041415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8116253b-35e9-4a02-a639-e9f8771c03e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:09.202341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3a1fbe02-498b-402c-a7f2-ba8389249be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:09.205586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3a1fbe02-498b-402c-a7f2-ba8389249be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:09.218680 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a1fbe02-498b-402c-a7f2-ba8389249be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:09.219607 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a1fbe02-498b-402c-a7f2-ba8389249be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:09.220325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a1fbe02-498b-402c-a7f2-ba8389249be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:09.221719 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a1fbe02-498b-402c-a7f2-ba8389249be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:09.222822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a1fbe02-498b-402c-a7f2-ba8389249be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:09.223544 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a1fbe02-498b-402c-a7f2-ba8389249be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:09.275716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8116253b-35e9-4a02-a639-e9f8771c03e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:09.285707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8116253b-35e9-4a02-a639-e9f8771c03e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.260713 Aug 07 01:13:09.286301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 189/391] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:09 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:09.477489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3a1fbe02-498b-402c-a7f2-ba8389249be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:09.497433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3a1fbe02-498b-402c-a7f2-ba8389249be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:09.502550 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3a1fbe02-498b-402c-a7f2-ba8389249be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:09.509082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3a1fbe02-498b-402c-a7f2-ba8389249be4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1752 microversion: 2.20 time: 0.310575 Aug 07 01:13:09.510449 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 203/392] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:09 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1752 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:10.347813 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b12e2590-a6f4-4e16-b32d-0bdb8efeeec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:10.350228 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b12e2590-a6f4-4e16-b32d-0bdb8efeeec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:10.367948 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b12e2590-a6f4-4e16-b32d-0bdb8efeeec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:10.367948 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b12e2590-a6f4-4e16-b32d-0bdb8efeeec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:10.367948 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b12e2590-a6f4-4e16-b32d-0bdb8efeeec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:10.368542 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b12e2590-a6f4-4e16-b32d-0bdb8efeeec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:10.369504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b12e2590-a6f4-4e16-b32d-0bdb8efeeec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:10.370293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b12e2590-a6f4-4e16-b32d-0bdb8efeeec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:10.538673 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3241997d-fa5d-4e56-980d-f50224db4f8f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:10.539730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3241997d-fa5d-4e56-980d-f50224db4f8f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:10.549623 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3241997d-fa5d-4e56-980d-f50224db4f8f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:10.550126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3241997d-fa5d-4e56-980d-f50224db4f8f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:10.550574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3241997d-fa5d-4e56-980d-f50224db4f8f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:10.551410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3241997d-fa5d-4e56-980d-f50224db4f8f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:10.551877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3241997d-fa5d-4e56-980d-f50224db4f8f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:10.552342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3241997d-fa5d-4e56-980d-f50224db4f8f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:10.602804 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b12e2590-a6f4-4e16-b32d-0bdb8efeeec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:10.616135 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b12e2590-a6f4-4e16-b32d-0bdb8efeeec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.275257 Aug 07 01:13:10.622474 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 190/393] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:10 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:10.771383 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3241997d-fa5d-4e56-980d-f50224db4f8f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:10.783908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3241997d-fa5d-4e56-980d-f50224db4f8f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:10.786453 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3241997d-fa5d-4e56-980d-f50224db4f8f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:10.793654 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3241997d-fa5d-4e56-980d-f50224db4f8f tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.259510 Aug 07 01:13:10.794676 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 204/394] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:10 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:13:11.648266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9b7e3706-cc14-448c-bc2a-358a2d866ab0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:11.649852 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9b7e3706-cc14-448c-bc2a-358a2d866ab0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:11.653449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=76 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:13:11.663334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9b7e3706-cc14-448c-bc2a-358a2d866ab0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:11.664900 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9b7e3706-cc14-448c-bc2a-358a2d866ab0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:11.665214 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9b7e3706-cc14-448c-bc2a-358a2d866ab0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:11.666118 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9b7e3706-cc14-448c-bc2a-358a2d866ab0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:11.666881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9b7e3706-cc14-448c-bc2a-358a2d866ab0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:11.667532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9b7e3706-cc14-448c-bc2a-358a2d866ab0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:11.826088 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-79888f80-ede2-417b-a5d7-c054d45faee1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:11.828382 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-79888f80-ede2-417b-a5d7-c054d45faee1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:11.841576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-79888f80-ede2-417b-a5d7-c054d45faee1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:11.842481 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-79888f80-ede2-417b-a5d7-c054d45faee1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:11.843870 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-79888f80-ede2-417b-a5d7-c054d45faee1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:11.845514 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-79888f80-ede2-417b-a5d7-c054d45faee1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:11.846589 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-79888f80-ede2-417b-a5d7-c054d45faee1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:11.847561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-79888f80-ede2-417b-a5d7-c054d45faee1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:12.008311 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9b7e3706-cc14-448c-bc2a-358a2d866ab0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:12.022102 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9b7e3706-cc14-448c-bc2a-358a2d866ab0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.379020 Aug 07 01:13:12.023496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 191/395] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:11 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:12.190389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-79888f80-ede2-417b-a5d7-c054d45faee1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:12.208580 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-79888f80-ede2-417b-a5d7-c054d45faee1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:12.211880 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-79888f80-ede2-417b-a5d7-c054d45faee1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:12.214249 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-79888f80-ede2-417b-a5d7-c054d45faee1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.396005 Aug 07 01:13:12.215267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 205/396] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:11 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:13:13.068779 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9dc371c5-d6e6-43b3-881f-04391757c692 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:13.072620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9dc371c5-d6e6-43b3-881f-04391757c692 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:13.104567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9dc371c5-d6e6-43b3-881f-04391757c692 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:13.105924 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9dc371c5-d6e6-43b3-881f-04391757c692 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:13.106806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9dc371c5-d6e6-43b3-881f-04391757c692 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:13.109349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9dc371c5-d6e6-43b3-881f-04391757c692 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:13.111461 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9dc371c5-d6e6-43b3-881f-04391757c692 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:13.115187 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9dc371c5-d6e6-43b3-881f-04391757c692 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:13.249504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2df4559e-1d0f-410f-9b3d-61dbd6aefcea tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:13.258433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2df4559e-1d0f-410f-9b3d-61dbd6aefcea tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:13.299361 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2df4559e-1d0f-410f-9b3d-61dbd6aefcea tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:13.300718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2df4559e-1d0f-410f-9b3d-61dbd6aefcea tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:13.301758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2df4559e-1d0f-410f-9b3d-61dbd6aefcea tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:13.303369 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2df4559e-1d0f-410f-9b3d-61dbd6aefcea tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:13.304445 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2df4559e-1d0f-410f-9b3d-61dbd6aefcea tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:13.305517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2df4559e-1d0f-410f-9b3d-61dbd6aefcea tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:13.593103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9dc371c5-d6e6-43b3-881f-04391757c692 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:13.610587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9dc371c5-d6e6-43b3-881f-04391757c692 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.553852 Aug 07 01:13:13.613741 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 192/397] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:13 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:13.735985 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2df4559e-1d0f-410f-9b3d-61dbd6aefcea tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:13.739409 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:13:13.753617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2df4559e-1d0f-410f-9b3d-61dbd6aefcea tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:13.756072 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2df4559e-1d0f-410f-9b3d-61dbd6aefcea tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:13.757672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2df4559e-1d0f-410f-9b3d-61dbd6aefcea tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.518991 Aug 07 01:13:13.759402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 206/398] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:13 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 523 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:14.661256 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-da7dd4f2-ef51-4c03-bc2e-672f904e9f63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:14.662100 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-da7dd4f2-ef51-4c03-bc2e-672f904e9f63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:14.677759 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-da7dd4f2-ef51-4c03-bc2e-672f904e9f63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:14.678330 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-da7dd4f2-ef51-4c03-bc2e-672f904e9f63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:14.678806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-da7dd4f2-ef51-4c03-bc2e-672f904e9f63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:14.679559 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-da7dd4f2-ef51-4c03-bc2e-672f904e9f63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:14.680025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-da7dd4f2-ef51-4c03-bc2e-672f904e9f63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:14.680487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-da7dd4f2-ef51-4c03-bc2e-672f904e9f63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:14.802932 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-59f16036-8c0e-48cc-ad21-35e3d577602c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:14.806138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-59f16036-8c0e-48cc-ad21-35e3d577602c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:14.833471 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-59f16036-8c0e-48cc-ad21-35e3d577602c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:14.836134 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-59f16036-8c0e-48cc-ad21-35e3d577602c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:14.837648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-59f16036-8c0e-48cc-ad21-35e3d577602c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:14.843874 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-59f16036-8c0e-48cc-ad21-35e3d577602c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:14.845115 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-59f16036-8c0e-48cc-ad21-35e3d577602c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:14.847697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-59f16036-8c0e-48cc-ad21-35e3d577602c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:15.006438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-da7dd4f2-ef51-4c03-bc2e-672f904e9f63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:15.018097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-da7dd4f2-ef51-4c03-bc2e-672f904e9f63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.360875 Aug 07 01:13:15.027371 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 193/399] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:14 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:15.186924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-59f16036-8c0e-48cc-ad21-35e3d577602c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:15.203863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-59f16036-8c0e-48cc-ad21-35e3d577602c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:15.206550 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-59f16036-8c0e-48cc-ad21-35e3d577602c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:15.208303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-59f16036-8c0e-48cc-ad21-35e3d577602c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.411223 Aug 07 01:13:15.225097 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 207/400] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:14 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:13:16.085685 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7ce56702-1ab2-4d3c-b192-3d523c429902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:16.098680 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7ce56702-1ab2-4d3c-b192-3d523c429902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:16.121218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7ce56702-1ab2-4d3c-b192-3d523c429902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:16.121719 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7ce56702-1ab2-4d3c-b192-3d523c429902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:16.122157 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7ce56702-1ab2-4d3c-b192-3d523c429902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:16.122884 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7ce56702-1ab2-4d3c-b192-3d523c429902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:16.123332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7ce56702-1ab2-4d3c-b192-3d523c429902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:16.123753 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7ce56702-1ab2-4d3c-b192-3d523c429902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:16.244946 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-da3877e5-2076-407d-bd0c-6429ea4f512c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:16.249661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-da3877e5-2076-407d-bd0c-6429ea4f512c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:16.273936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-da3877e5-2076-407d-bd0c-6429ea4f512c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:16.274762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-da3877e5-2076-407d-bd0c-6429ea4f512c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:16.275568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-da3877e5-2076-407d-bd0c-6429ea4f512c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:16.276912 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-da3877e5-2076-407d-bd0c-6429ea4f512c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:16.277761 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-da3877e5-2076-407d-bd0c-6429ea4f512c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:16.278518 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-da3877e5-2076-407d-bd0c-6429ea4f512c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:16.382911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7ce56702-1ab2-4d3c-b192-3d523c429902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:16.411602 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7ce56702-1ab2-4d3c-b192-3d523c429902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.330807 Aug 07 01:13:16.428527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 194/401] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:16 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:16.897360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-da3877e5-2076-407d-bd0c-6429ea4f512c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:16.930210 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-da3877e5-2076-407d-bd0c-6429ea4f512c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:16.934374 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-da3877e5-2076-407d-bd0c-6429ea4f512c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:16.939982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-da3877e5-2076-407d-bd0c-6429ea4f512c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.706109 Aug 07 01:13:16.942794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 208/402] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:16 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:17.451255 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-04d4cb25-e235-4e6f-b481-b74df80fa992 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:17.453367 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-04d4cb25-e235-4e6f-b481-b74df80fa992 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:17.462922 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-04d4cb25-e235-4e6f-b481-b74df80fa992 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:17.463776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-04d4cb25-e235-4e6f-b481-b74df80fa992 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:17.464494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-04d4cb25-e235-4e6f-b481-b74df80fa992 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:17.465517 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-04d4cb25-e235-4e6f-b481-b74df80fa992 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:17.466005 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-04d4cb25-e235-4e6f-b481-b74df80fa992 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:17.466520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-04d4cb25-e235-4e6f-b481-b74df80fa992 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:17.730479 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-04d4cb25-e235-4e6f-b481-b74df80fa992 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:17.750227 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-04d4cb25-e235-4e6f-b481-b74df80fa992 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.301670 Aug 07 01:13:17.753318 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 195/403] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:17 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:17.978941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8da8c9e1-02b2-4a43-b97e-3b84c621e800 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:17.984094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8da8c9e1-02b2-4a43-b97e-3b84c621e800 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:17.998440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8da8c9e1-02b2-4a43-b97e-3b84c621e800 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:18.000458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8da8c9e1-02b2-4a43-b97e-3b84c621e800 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:18.001869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8da8c9e1-02b2-4a43-b97e-3b84c621e800 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:18.003670 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8da8c9e1-02b2-4a43-b97e-3b84c621e800 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:18.005113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8da8c9e1-02b2-4a43-b97e-3b84c621e800 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:18.006279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8da8c9e1-02b2-4a43-b97e-3b84c621e800 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:18.344530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8da8c9e1-02b2-4a43-b97e-3b84c621e800 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:18.391755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8da8c9e1-02b2-4a43-b97e-3b84c621e800 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:18.397816 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8da8c9e1-02b2-4a43-b97e-3b84c621e800 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:18.402730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8da8c9e1-02b2-4a43-b97e-3b84c621e800 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.430038 Aug 07 01:13:18.417101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 209/404] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:17 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:18.799986 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2231af7d-454a-4c34-bbb6-e2f8cedbf9b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:18.801796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2231af7d-454a-4c34-bbb6-e2f8cedbf9b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:18.814061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2231af7d-454a-4c34-bbb6-e2f8cedbf9b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:18.815781 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2231af7d-454a-4c34-bbb6-e2f8cedbf9b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:18.817015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2231af7d-454a-4c34-bbb6-e2f8cedbf9b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:18.818898 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2231af7d-454a-4c34-bbb6-e2f8cedbf9b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:18.819761 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2231af7d-454a-4c34-bbb6-e2f8cedbf9b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:18.820862 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2231af7d-454a-4c34-bbb6-e2f8cedbf9b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:19.120917 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2231af7d-454a-4c34-bbb6-e2f8cedbf9b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:19.143154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2231af7d-454a-4c34-bbb6-e2f8cedbf9b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.350188 Aug 07 01:13:19.145531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 196/405] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:18 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:19.484548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-14a4ed47-6d56-48a7-aaf0-a6e053a6eb45 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:19.485880 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-14a4ed47-6d56-48a7-aaf0-a6e053a6eb45 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:19.492887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=91,nova_api:UPDATE=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:13:19.501252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-14a4ed47-6d56-48a7-aaf0-a6e053a6eb45 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:19.502315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-14a4ed47-6d56-48a7-aaf0-a6e053a6eb45 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:19.504813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-14a4ed47-6d56-48a7-aaf0-a6e053a6eb45 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:19.507425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-14a4ed47-6d56-48a7-aaf0-a6e053a6eb45 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:19.508613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-14a4ed47-6d56-48a7-aaf0-a6e053a6eb45 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:19.510098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-14a4ed47-6d56-48a7-aaf0-a6e053a6eb45 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:19.975184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-14a4ed47-6d56-48a7-aaf0-a6e053a6eb45 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:19.993749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-14a4ed47-6d56-48a7-aaf0-a6e053a6eb45 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:20.001915 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-14a4ed47-6d56-48a7-aaf0-a6e053a6eb45 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:20.008422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-14a4ed47-6d56-48a7-aaf0-a6e053a6eb45 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.542352 Aug 07 01:13:20.009968 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 210/406] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:19 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:13:20.187589 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1e04ab0f-b189-47dd-85f8-27927e626d5a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:20.189498 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1e04ab0f-b189-47dd-85f8-27927e626d5a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:20.202887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e04ab0f-b189-47dd-85f8-27927e626d5a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:20.203725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e04ab0f-b189-47dd-85f8-27927e626d5a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:20.205363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e04ab0f-b189-47dd-85f8-27927e626d5a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:20.206378 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e04ab0f-b189-47dd-85f8-27927e626d5a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:20.207223 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e04ab0f-b189-47dd-85f8-27927e626d5a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:20.207841 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e04ab0f-b189-47dd-85f8-27927e626d5a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:20.575663 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1e04ab0f-b189-47dd-85f8-27927e626d5a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:20.611446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1e04ab0f-b189-47dd-85f8-27927e626d5a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.428587 Aug 07 01:13:20.631215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 197/407] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:20 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:21.067226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-855a325a-4ab6-4f3d-94fb-4d67b12a9c70 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:21.067863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-855a325a-4ab6-4f3d-94fb-4d67b12a9c70 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:21.086987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-855a325a-4ab6-4f3d-94fb-4d67b12a9c70 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:21.087896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-855a325a-4ab6-4f3d-94fb-4d67b12a9c70 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:21.096849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-855a325a-4ab6-4f3d-94fb-4d67b12a9c70 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:21.098222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-855a325a-4ab6-4f3d-94fb-4d67b12a9c70 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:21.098905 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-855a325a-4ab6-4f3d-94fb-4d67b12a9c70 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:21.099372 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-855a325a-4ab6-4f3d-94fb-4d67b12a9c70 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:21.392144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-855a325a-4ab6-4f3d-94fb-4d67b12a9c70 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:21.412988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-855a325a-4ab6-4f3d-94fb-4d67b12a9c70 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:21.426154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-855a325a-4ab6-4f3d-94fb-4d67b12a9c70 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:21.429426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-855a325a-4ab6-4f3d-94fb-4d67b12a9c70 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.366869 Aug 07 01:13:21.440310 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 211/408] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:21 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:21.646926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2aade222-d009-4327-8caf-78bdcd73ff73 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:21.647917 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2aade222-d009-4327-8caf-78bdcd73ff73 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:21.698843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2aade222-d009-4327-8caf-78bdcd73ff73 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:21.700261 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2aade222-d009-4327-8caf-78bdcd73ff73 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:21.701390 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2aade222-d009-4327-8caf-78bdcd73ff73 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:21.703021 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2aade222-d009-4327-8caf-78bdcd73ff73 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:21.706629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2aade222-d009-4327-8caf-78bdcd73ff73 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:21.708018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2aade222-d009-4327-8caf-78bdcd73ff73 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:22.221003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2aade222-d009-4327-8caf-78bdcd73ff73 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:22.250953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2aade222-d009-4327-8caf-78bdcd73ff73 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.613508 Aug 07 01:13:22.251837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 198/409] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:21 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 616 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:22.475286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-085c37f7-af7e-4cf6-98a2-6c264583c1dd tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:22.479230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-085c37f7-af7e-4cf6-98a2-6c264583c1dd tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:22.495581 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-085c37f7-af7e-4cf6-98a2-6c264583c1dd tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:22.497797 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-085c37f7-af7e-4cf6-98a2-6c264583c1dd tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:22.499407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-085c37f7-af7e-4cf6-98a2-6c264583c1dd tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:22.503742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-085c37f7-af7e-4cf6-98a2-6c264583c1dd tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:22.505118 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-085c37f7-af7e-4cf6-98a2-6c264583c1dd tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:22.506508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-085c37f7-af7e-4cf6-98a2-6c264583c1dd tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:22.911094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-085c37f7-af7e-4cf6-98a2-6c264583c1dd tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:22.926548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-085c37f7-af7e-4cf6-98a2-6c264583c1dd tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:22.928902 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-085c37f7-af7e-4cf6-98a2-6c264583c1dd tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:22.930515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-085c37f7-af7e-4cf6-98a2-6c264583c1dd tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.472348 Aug 07 01:13:22.931358 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 212/410] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:22 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:23.294129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e57be051-f004-4054-a4bf-bfc7ff45d766 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:23.295372 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e57be051-f004-4054-a4bf-bfc7ff45d766 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:23.311425 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e57be051-f004-4054-a4bf-bfc7ff45d766 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:23.312422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e57be051-f004-4054-a4bf-bfc7ff45d766 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:23.313202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e57be051-f004-4054-a4bf-bfc7ff45d766 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:23.314486 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e57be051-f004-4054-a4bf-bfc7ff45d766 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:23.315256 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e57be051-f004-4054-a4bf-bfc7ff45d766 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:23.316034 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e57be051-f004-4054-a4bf-bfc7ff45d766 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:23.523020 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e57be051-f004-4054-a4bf-bfc7ff45d766 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:23.531959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e57be051-f004-4054-a4bf-bfc7ff45d766 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.246585 Aug 07 01:13:23.539380 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 199/411] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:23 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:23.976674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d2ac1ecc-e215-448f-b3a0-62e5d0223ef9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:23.977280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d2ac1ecc-e215-448f-b3a0-62e5d0223ef9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:23.984796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d2ac1ecc-e215-448f-b3a0-62e5d0223ef9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:23.985386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d2ac1ecc-e215-448f-b3a0-62e5d0223ef9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:23.986081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d2ac1ecc-e215-448f-b3a0-62e5d0223ef9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:23.986836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d2ac1ecc-e215-448f-b3a0-62e5d0223ef9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:23.987302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d2ac1ecc-e215-448f-b3a0-62e5d0223ef9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:23.988105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d2ac1ecc-e215-448f-b3a0-62e5d0223ef9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:24.149937 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-667428070", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "networks": [{"uuid": "ea640be5-e5dd-479f-a491-fb3ae7cc114e"}]}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:24.249648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d2ac1ecc-e215-448f-b3a0-62e5d0223ef9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:24.251022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.network.neutron [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] validate_networks() for [('ea640be5-e5dd-479f-a491-fb3ae7cc114e', None, None, None, None, None)] {{(pid=117984) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:13:24.264681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d2ac1ecc-e215-448f-b3a0-62e5d0223ef9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:24.270394 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d2ac1ecc-e215-448f-b3a0-62e5d0223ef9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:24.272788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d2ac1ecc-e215-448f-b3a0-62e5d0223ef9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.302148 Aug 07 01:13:24.274006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 213/412] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:23 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:13:24.557693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d1471781-6540-4ad2-a483-765158befd21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:24.559003 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d1471781-6540-4ad2-a483-765158befd21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:24.567111 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1471781-6540-4ad2-a483-765158befd21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:24.567705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1471781-6540-4ad2-a483-765158befd21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:24.568343 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1471781-6540-4ad2-a483-765158befd21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:24.569545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1471781-6540-4ad2-a483-765158befd21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:24.570085 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1471781-6540-4ad2-a483-765158befd21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:24.570835 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1471781-6540-4ad2-a483-765158befd21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:24.796221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d1471781-6540-4ad2-a483-765158befd21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:24.810283 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d1471781-6540-4ad2-a483-765158befd21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.257284 Aug 07 01:13:24.815174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 214/413] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:24 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:25.119898 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Flavor limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:13:25.120833 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Image limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:13:25.122183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Flavor pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:13:25.123267 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Image pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:13:25.124133 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:13:25.145406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:25.146369 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:25.147216 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:25.148656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:25.149632 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:25.150371 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:25.192691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Getting quotas for project f7b08cc6d4a34961bdf464b58946a02e. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:13:25.207614 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Getting quotas for user 6dd0ed2144164190b380ed58b3dd7cd4 and project f7b08cc6d4a34961bdf464b58946a02e. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:13:25.240911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Going to run 1 instances... {{(pid=117984) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:13:25.246001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117984) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:13:25.299853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-61800d32-63a9-47f3-a566-e9a2d4295ef1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:25.301176 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-61800d32-63a9-47f3-a566-e9a2d4295ef1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:25.309166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61800d32-63a9-47f3-a566-e9a2d4295ef1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:25.310396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61800d32-63a9-47f3-a566-e9a2d4295ef1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:25.311026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61800d32-63a9-47f3-a566-e9a2d4295ef1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:25.311993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61800d32-63a9-47f3-a566-e9a2d4295ef1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:25.312708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61800d32-63a9-47f3-a566-e9a2d4295ef1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:25.313339 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61800d32-63a9-47f3-a566-e9a2d4295ef1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:25.380467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c51cfe5b-3412-4f32-9597-faf41d792475 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.237922 Aug 07 01:13:25.384066 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 200/414] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:13:24 2022] POST /compute/v2.1/servers => generated 380 bytes in 1241 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:13:25.403817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-89067c78-b78c-4481-80fd-f7229801e3af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:25.405578 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-89067c78-b78c-4481-80fd-f7229801e3af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:25.572917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-61800d32-63a9-47f3-a566-e9a2d4295ef1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:25.593957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-61800d32-63a9-47f3-a566-e9a2d4295ef1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:25.599646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-61800d32-63a9-47f3-a566-e9a2d4295ef1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:25.602892 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-61800d32-63a9-47f3-a566-e9a2d4295ef1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.307218 Aug 07 01:13:25.619051 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 215/415] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:25 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:25.660411 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-89067c78-b78c-4481-80fd-f7229801e3af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:25.675559 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-89067c78-b78c-4481-80fd-f7229801e3af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1218 microversion: 2.1 time: 0.278850 Aug 07 01:13:25.698449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 201/416] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:25 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1218 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:25.838738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-084029cb-37e5-4ad4-bd33-9821745de50c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:25.840493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-084029cb-37e5-4ad4-bd33-9821745de50c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:25.852016 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-084029cb-37e5-4ad4-bd33-9821745de50c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:25.853596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-084029cb-37e5-4ad4-bd33-9821745de50c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:25.854953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-084029cb-37e5-4ad4-bd33-9821745de50c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:25.856643 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-084029cb-37e5-4ad4-bd33-9821745de50c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:25.859713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-084029cb-37e5-4ad4-bd33-9821745de50c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:25.859713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-084029cb-37e5-4ad4-bd33-9821745de50c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:26.153442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-084029cb-37e5-4ad4-bd33-9821745de50c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:26.165856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-084029cb-37e5-4ad4-bd33-9821745de50c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1522 microversion: 2.1 time: 0.334431 Aug 07 01:13:26.167666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 216/417] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:25 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1522 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:26.631007 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7911643b-ad30-4d73-8b56-4ee44d2794ef tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:26.631849 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7911643b-ad30-4d73-8b56-4ee44d2794ef tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:26.652024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7911643b-ad30-4d73-8b56-4ee44d2794ef tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:26.652902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7911643b-ad30-4d73-8b56-4ee44d2794ef tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:26.653506 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7911643b-ad30-4d73-8b56-4ee44d2794ef tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:26.654285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7911643b-ad30-4d73-8b56-4ee44d2794ef tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:26.654757 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7911643b-ad30-4d73-8b56-4ee44d2794ef tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:26.655208 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7911643b-ad30-4d73-8b56-4ee44d2794ef tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:26.707734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-93cdb86c-fce8-401b-80e3-dd391d75981c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:26.709206 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-93cdb86c-fce8-401b-80e3-dd391d75981c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:26.722791 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-93cdb86c-fce8-401b-80e3-dd391d75981c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:26.723562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-93cdb86c-fce8-401b-80e3-dd391d75981c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:26.729019 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-93cdb86c-fce8-401b-80e3-dd391d75981c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:26.729019 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-93cdb86c-fce8-401b-80e3-dd391d75981c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:26.730485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-93cdb86c-fce8-401b-80e3-dd391d75981c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:26.731604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-93cdb86c-fce8-401b-80e3-dd391d75981c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:26.906347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-93cdb86c-fce8-401b-80e3-dd391d75981c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:26.918534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-93cdb86c-fce8-401b-80e3-dd391d75981c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1214 microversion: 2.1 time: 0.216116 Aug 07 01:13:26.932455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 217/418] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:26 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1214 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:26.999456 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7911643b-ad30-4d73-8b56-4ee44d2794ef tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:27.033749 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7911643b-ad30-4d73-8b56-4ee44d2794ef tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:27.035758 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7911643b-ad30-4d73-8b56-4ee44d2794ef tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:27.038611 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7911643b-ad30-4d73-8b56-4ee44d2794ef tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.412138 Aug 07 01:13:27.039226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 202/419] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:26 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:13:27.204458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8f8e1518-19cb-4f40-b2cb-539e683b7d8a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:27.205364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8f8e1518-19cb-4f40-b2cb-539e683b7d8a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:27.213542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f8e1518-19cb-4f40-b2cb-539e683b7d8a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:27.214193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f8e1518-19cb-4f40-b2cb-539e683b7d8a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:27.214748 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f8e1518-19cb-4f40-b2cb-539e683b7d8a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:27.216477 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f8e1518-19cb-4f40-b2cb-539e683b7d8a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:27.217067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f8e1518-19cb-4f40-b2cb-539e683b7d8a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:27.217901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f8e1518-19cb-4f40-b2cb-539e683b7d8a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:27.470116 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8f8e1518-19cb-4f40-b2cb-539e683b7d8a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:27.480530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8f8e1518-19cb-4f40-b2cb-539e683b7d8a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1502 microversion: 2.1 time: 0.282203 Aug 07 01:13:27.481462 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 218/420] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:27 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1502 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:27.505649 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-33ba7a7b-5391-40ea-9706-1b90cf764a0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:27.506469 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-33ba7a7b-5391-40ea-9706-1b90cf764a0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:27.513848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-33ba7a7b-5391-40ea-9706-1b90cf764a0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:27.514492 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-33ba7a7b-5391-40ea-9706-1b90cf764a0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:27.515082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-33ba7a7b-5391-40ea-9706-1b90cf764a0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:27.515967 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-33ba7a7b-5391-40ea-9706-1b90cf764a0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:27.516603 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-33ba7a7b-5391-40ea-9706-1b90cf764a0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:27.517290 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-33ba7a7b-5391-40ea-9706-1b90cf764a0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:27.863099 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-33ba7a7b-5391-40ea-9706-1b90cf764a0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:27.877618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-33ba7a7b-5391-40ea-9706-1b90cf764a0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1517 microversion: 2.1 time: 0.378078 Aug 07 01:13:27.878628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 203/421] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:27 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1517 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:27.953341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-49aa8cff-efb9-4f65-b615-6cf8d6199ab3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:27.954564 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-49aa8cff-efb9-4f65-b615-6cf8d6199ab3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:27.968183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49aa8cff-efb9-4f65-b615-6cf8d6199ab3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:27.969066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49aa8cff-efb9-4f65-b615-6cf8d6199ab3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:27.969616 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49aa8cff-efb9-4f65-b615-6cf8d6199ab3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:27.971035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49aa8cff-efb9-4f65-b615-6cf8d6199ab3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:27.971777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49aa8cff-efb9-4f65-b615-6cf8d6199ab3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:27.972551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49aa8cff-efb9-4f65-b615-6cf8d6199ab3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:28.098380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-49aa8cff-efb9-4f65-b615-6cf8d6199ab3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:28.121451 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-52e287fa-94da-4d6c-94bd-fecb4eb676a7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:28.123080 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-52e287fa-94da-4d6c-94bd-fecb4eb676a7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:28.132114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-49aa8cff-efb9-4f65-b615-6cf8d6199ab3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1270 microversion: 2.1 time: 0.189666 Aug 07 01:13:28.137045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 219/422] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:27 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1270 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:28.152551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-52e287fa-94da-4d6c-94bd-fecb4eb676a7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:28.153681 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-52e287fa-94da-4d6c-94bd-fecb4eb676a7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:28.155905 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-52e287fa-94da-4d6c-94bd-fecb4eb676a7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:28.157105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-52e287fa-94da-4d6c-94bd-fecb4eb676a7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:28.159035 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-52e287fa-94da-4d6c-94bd-fecb4eb676a7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:28.160108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-52e287fa-94da-4d6c-94bd-fecb4eb676a7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:29.116859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-52e287fa-94da-4d6c-94bd-fecb4eb676a7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:29.135421 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-52e287fa-94da-4d6c-94bd-fecb4eb676a7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:29.139404 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-52e287fa-94da-4d6c-94bd-fecb4eb676a7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:29.144326 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-52e287fa-94da-4d6c-94bd-fecb4eb676a7 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 1.074890 Aug 07 01:13:29.146102 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 204/423] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:28 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 1078 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:29.184642 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d29babd0-b6dd-4203-99e0-2fd579779fba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:29.185548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d29babd0-b6dd-4203-99e0-2fd579779fba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:29.193182 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d29babd0-b6dd-4203-99e0-2fd579779fba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:29.193670 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d29babd0-b6dd-4203-99e0-2fd579779fba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:29.194311 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d29babd0-b6dd-4203-99e0-2fd579779fba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:29.195124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d29babd0-b6dd-4203-99e0-2fd579779fba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:29.195886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d29babd0-b6dd-4203-99e0-2fd579779fba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:29.196346 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d29babd0-b6dd-4203-99e0-2fd579779fba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:29.276875 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d29babd0-b6dd-4203-99e0-2fd579779fba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:29.294627 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d29babd0-b6dd-4203-99e0-2fd579779fba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1276 microversion: 2.1 time: 0.128227 Aug 07 01:13:29.304296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 220/424] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:29 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1276 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:29.876776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4ae3ea68-31ee-40f2-b68a-defcc6a61b4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:29.881969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4ae3ea68-31ee-40f2-b68a-defcc6a61b4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:29.895097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ae3ea68-31ee-40f2-b68a-defcc6a61b4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:29.895927 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ae3ea68-31ee-40f2-b68a-defcc6a61b4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:29.896524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ae3ea68-31ee-40f2-b68a-defcc6a61b4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:29.899266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ae3ea68-31ee-40f2-b68a-defcc6a61b4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:29.899266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ae3ea68-31ee-40f2-b68a-defcc6a61b4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:29.900744 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ae3ea68-31ee-40f2-b68a-defcc6a61b4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:29.934653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.objects.instance [None req-4ae3ea68-31ee-40f2-b68a-defcc6a61b4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lazy-loading 'flavor' on Instance uuid 99e925dd-a3a5-44ab-bf20-9c80807d5530 {{(pid=117984) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:13:29.941075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:13:30.069608 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4ae3ea68-31ee-40f2-b68a-defcc6a61b4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "POST /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530/action" status: 200 len: 32 microversion: 2.1 time: 0.197582 Aug 07 01:13:30.077853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 205/425] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:13:29 2022] POST /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530/action => generated 32 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Aug 07 01:13:30.089173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2dbc02fe-1506-49fa-be47-00a9850cb20b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:30.090716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2dbc02fe-1506-49fa-be47-00a9850cb20b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:30.102079 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2dbc02fe-1506-49fa-be47-00a9850cb20b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:30.103364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2dbc02fe-1506-49fa-be47-00a9850cb20b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:30.104568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2dbc02fe-1506-49fa-be47-00a9850cb20b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:30.106218 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2dbc02fe-1506-49fa-be47-00a9850cb20b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:30.107407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2dbc02fe-1506-49fa-be47-00a9850cb20b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:30.108536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2dbc02fe-1506-49fa-be47-00a9850cb20b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:30.193216 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ecdd6b1f-d26b-4ef4-8154-09fc206c16f9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:30.198138 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ecdd6b1f-d26b-4ef4-8154-09fc206c16f9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:30.212177 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecdd6b1f-d26b-4ef4-8154-09fc206c16f9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:30.213466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecdd6b1f-d26b-4ef4-8154-09fc206c16f9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:30.214518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecdd6b1f-d26b-4ef4-8154-09fc206c16f9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:30.216168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecdd6b1f-d26b-4ef4-8154-09fc206c16f9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:30.217820 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecdd6b1f-d26b-4ef4-8154-09fc206c16f9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:30.218612 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecdd6b1f-d26b-4ef4-8154-09fc206c16f9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:30.598962 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2dbc02fe-1506-49fa-be47-00a9850cb20b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:30.623401 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2dbc02fe-1506-49fa-be47-00a9850cb20b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.540303 Aug 07 01:13:30.627611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 221/426] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:30 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:30.640633 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ecdd6b1f-d26b-4ef4-8154-09fc206c16f9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:30.660993 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ecdd6b1f-d26b-4ef4-8154-09fc206c16f9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:30.669933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ecdd6b1f-d26b-4ef4-8154-09fc206c16f9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:30.682719 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ecdd6b1f-d26b-4ef4-8154-09fc206c16f9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.497571 Aug 07 01:13:30.683925 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 206/427] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:30 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:13:30.707414 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-54dc2336-8279-49e2-b352-21501bd3dbd2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:30.708773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-54dc2336-8279-49e2-b352-21501bd3dbd2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:30.731600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-54dc2336-8279-49e2-b352-21501bd3dbd2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:30.732385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-54dc2336-8279-49e2-b352-21501bd3dbd2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:30.733049 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-54dc2336-8279-49e2-b352-21501bd3dbd2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:30.734490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-54dc2336-8279-49e2-b352-21501bd3dbd2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:30.736445 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-54dc2336-8279-49e2-b352-21501bd3dbd2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:30.737364 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-54dc2336-8279-49e2-b352-21501bd3dbd2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:30.926680 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-54dc2336-8279-49e2-b352-21501bd3dbd2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:30.959112 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-54dc2336-8279-49e2-b352-21501bd3dbd2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1276 microversion: 2.1 time: 0.272934 Aug 07 01:13:30.962213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 207/428] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:30 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1276 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:31.688506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ec3b16b7-6652-4513-8e1e-7badbac35797 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:31.691457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ec3b16b7-6652-4513-8e1e-7badbac35797 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:31.704072 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec3b16b7-6652-4513-8e1e-7badbac35797 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:31.706193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec3b16b7-6652-4513-8e1e-7badbac35797 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:31.708031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec3b16b7-6652-4513-8e1e-7badbac35797 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:31.712855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec3b16b7-6652-4513-8e1e-7badbac35797 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:31.714154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec3b16b7-6652-4513-8e1e-7badbac35797 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:31.715327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec3b16b7-6652-4513-8e1e-7badbac35797 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:31.737463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1b28b825-6ff7-46dd-a6f5-64c9c89bea12 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:31.739245 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1b28b825-6ff7-46dd-a6f5-64c9c89bea12 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:31.770258 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b28b825-6ff7-46dd-a6f5-64c9c89bea12 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:31.771087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b28b825-6ff7-46dd-a6f5-64c9c89bea12 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:31.771890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b28b825-6ff7-46dd-a6f5-64c9c89bea12 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:31.774318 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b28b825-6ff7-46dd-a6f5-64c9c89bea12 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:31.776714 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b28b825-6ff7-46dd-a6f5-64c9c89bea12 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:31.778259 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b28b825-6ff7-46dd-a6f5-64c9c89bea12 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:32.027693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ec3b16b7-6652-4513-8e1e-7badbac35797 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:32.042586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ec3b16b7-6652-4513-8e1e-7badbac35797 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.359354 Aug 07 01:13:32.047493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 222/429] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:31 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:32.063444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1259a7ce-81ff-4c2e-9918-27e262659417 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:32.065318 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1259a7ce-81ff-4c2e-9918-27e262659417 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:32.078558 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1259a7ce-81ff-4c2e-9918-27e262659417 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:32.079532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1259a7ce-81ff-4c2e-9918-27e262659417 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:32.080844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1259a7ce-81ff-4c2e-9918-27e262659417 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:32.082304 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1259a7ce-81ff-4c2e-9918-27e262659417 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:32.083069 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1259a7ce-81ff-4c2e-9918-27e262659417 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:32.083889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1259a7ce-81ff-4c2e-9918-27e262659417 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:32.120176 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:13:32.357613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1259a7ce-81ff-4c2e-9918-27e262659417 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:32.369961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1259a7ce-81ff-4c2e-9918-27e262659417 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1318 microversion: 2.1 time: 0.321160 Aug 07 01:13:32.371316 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 223/430] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:32 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1318 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:32.401539 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1b28b825-6ff7-46dd-a6f5-64c9c89bea12 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:32.414550 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1b28b825-6ff7-46dd-a6f5-64c9c89bea12 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:32.416723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1b28b825-6ff7-46dd-a6f5-64c9c89bea12 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:32.418497 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1b28b825-6ff7-46dd-a6f5-64c9c89bea12 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.688435 Aug 07 01:13:32.419626 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 208/431] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:31 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 690 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:13:33.085696 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ac352f30-9874-4215-b046-f517498276a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:33.087386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ac352f30-9874-4215-b046-f517498276a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:33.108582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac352f30-9874-4215-b046-f517498276a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:33.109520 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac352f30-9874-4215-b046-f517498276a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:33.110333 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac352f30-9874-4215-b046-f517498276a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:33.115781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac352f30-9874-4215-b046-f517498276a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:33.117923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac352f30-9874-4215-b046-f517498276a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:33.118757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac352f30-9874-4215-b046-f517498276a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:33.351415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ac352f30-9874-4215-b046-f517498276a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:33.380392 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ac352f30-9874-4215-b046-f517498276a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.302024 Aug 07 01:13:33.382394 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 224/432] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:33 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:33.407460 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-12ec7a26-3af7-493a-88b4-edbc3fc08242 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:33.409754 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-12ec7a26-3af7-493a-88b4-edbc3fc08242 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:33.418938 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-12ec7a26-3af7-493a-88b4-edbc3fc08242 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:33.420042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-12ec7a26-3af7-493a-88b4-edbc3fc08242 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:33.420757 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-12ec7a26-3af7-493a-88b4-edbc3fc08242 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:33.421940 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-12ec7a26-3af7-493a-88b4-edbc3fc08242 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:33.422718 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-12ec7a26-3af7-493a-88b4-edbc3fc08242 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:33.423437 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-12ec7a26-3af7-493a-88b4-edbc3fc08242 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:33.470433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-809c7d9d-4a2e-4546-a490-2f135c7da782 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:33.471428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-809c7d9d-4a2e-4546-a490-2f135c7da782 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:33.489239 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-809c7d9d-4a2e-4546-a490-2f135c7da782 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:33.492461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-809c7d9d-4a2e-4546-a490-2f135c7da782 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:33.493250 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-809c7d9d-4a2e-4546-a490-2f135c7da782 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:33.494421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-809c7d9d-4a2e-4546-a490-2f135c7da782 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:33.495676 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-809c7d9d-4a2e-4546-a490-2f135c7da782 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:33.497380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-809c7d9d-4a2e-4546-a490-2f135c7da782 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:33.753354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-809c7d9d-4a2e-4546-a490-2f135c7da782 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:33.772875 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-809c7d9d-4a2e-4546-a490-2f135c7da782 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:33.776323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-809c7d9d-4a2e-4546-a490-2f135c7da782 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:33.778729 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-809c7d9d-4a2e-4546-a490-2f135c7da782 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.316566 Aug 07 01:13:33.779769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 225/433] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:33 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:33.832892 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-12ec7a26-3af7-493a-88b4-edbc3fc08242 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:33.848882 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-12ec7a26-3af7-493a-88b4-edbc3fc08242 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1318 microversion: 2.1 time: 0.445680 Aug 07 01:13:33.850008 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 209/434] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:33 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1318 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:34.158776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:13:34.159585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:13:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:13:34.398681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-a04518fe-3efc-4952-9502-ec0e09e1ee1c req-a3afd8f0-61c4-482c-946e-af713a24da53 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc", "tag": "3ba58955-f906-4b1b-8224-8a7408f3a52c"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:34.430631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1e42909a-514a-4afe-9571-821d5f5e47d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:34.434390 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1e42909a-514a-4afe-9571-821d5f5e47d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:34.449908 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e42909a-514a-4afe-9571-821d5f5e47d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:34.450341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a04518fe-3efc-4952-9502-ec0e09e1ee1c req-a3afd8f0-61c4-482c-946e-af713a24da53 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:34.452135 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a04518fe-3efc-4952-9502-ec0e09e1ee1c req-a3afd8f0-61c4-482c-946e-af713a24da53 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:34.452712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a04518fe-3efc-4952-9502-ec0e09e1ee1c req-a3afd8f0-61c4-482c-946e-af713a24da53 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:34.453292 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e42909a-514a-4afe-9571-821d5f5e47d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:34.454056 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e42909a-514a-4afe-9571-821d5f5e47d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:34.456290 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e42909a-514a-4afe-9571-821d5f5e47d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:34.457331 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e42909a-514a-4afe-9571-821d5f5e47d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:34.458066 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e42909a-514a-4afe-9571-821d5f5e47d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:34.495225 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-a04518fe-3efc-4952-9502-ec0e09e1ee1c req-a3afd8f0-61c4-482c-946e-af713a24da53 service nova] Creating event network-changed:3ba58955-f906-4b1b-8224-8a7408f3a52c for instance 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc on hv-cinder-maste Aug 07 01:13:34.532364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-a04518fe-3efc-4952-9502-ec0e09e1ee1c req-a3afd8f0-61c4-482c-946e-af713a24da53 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.141231 Aug 07 01:13:34.534033 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 226/435] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:13:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:13:34.692570 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-3360a817-597d-490c-b715-c8179abe5b10 req-e028eddd-018a-48f1-9c8f-e2b54114c360 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7f60fb22-d17a-4b7c-9b7d-95543be1907e", "name": "network-vif-plugged", "status": "completed", "tag": "9ab59ce3-ed0d-4a8e-b76b-adeff559725a"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:34.713599 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-3360a817-597d-490c-b715-c8179abe5b10 req-e028eddd-018a-48f1-9c8f-e2b54114c360 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:34.714590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-3360a817-597d-490c-b715-c8179abe5b10 req-e028eddd-018a-48f1-9c8f-e2b54114c360 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:34.715294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-3360a817-597d-490c-b715-c8179abe5b10 req-e028eddd-018a-48f1-9c8f-e2b54114c360 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:34.731188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-3360a817-597d-490c-b715-c8179abe5b10 req-e028eddd-018a-48f1-9c8f-e2b54114c360 service nova] Creating event network-vif-plugged:9ab59ce3-ed0d-4a8e-b76b-adeff559725a for instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e on hv-cinder-maste Aug 07 01:13:34.738798 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-3360a817-597d-490c-b715-c8179abe5b10 req-e028eddd-018a-48f1-9c8f-e2b54114c360 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056996 Aug 07 01:13:34.740281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 227/436] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:13:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:13:34.832007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8716340a-f1a1-45d4-91d3-a61f85960ddf tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:34.832868 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8716340a-f1a1-45d4-91d3-a61f85960ddf tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:34.848013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8716340a-f1a1-45d4-91d3-a61f85960ddf tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:34.848561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8716340a-f1a1-45d4-91d3-a61f85960ddf tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:34.854167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8716340a-f1a1-45d4-91d3-a61f85960ddf tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:34.854987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8716340a-f1a1-45d4-91d3-a61f85960ddf tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:34.857891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8716340a-f1a1-45d4-91d3-a61f85960ddf tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:34.858360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8716340a-f1a1-45d4-91d3-a61f85960ddf tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:34.986011 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1e42909a-514a-4afe-9571-821d5f5e47d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:35.001070 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1e42909a-514a-4afe-9571-821d5f5e47d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.583103 Aug 07 01:13:35.002322 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 210/437] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:34 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:35.018963 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-39ab40f1-765b-4e4d-b952-ec34b6982c31 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:35.023921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-39ab40f1-765b-4e4d-b952-ec34b6982c31 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:35.040647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39ab40f1-765b-4e4d-b952-ec34b6982c31 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:35.041247 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39ab40f1-765b-4e4d-b952-ec34b6982c31 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:35.041706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39ab40f1-765b-4e4d-b952-ec34b6982c31 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:35.042466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39ab40f1-765b-4e4d-b952-ec34b6982c31 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:35.051967 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39ab40f1-765b-4e4d-b952-ec34b6982c31 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:35.053339 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39ab40f1-765b-4e4d-b952-ec34b6982c31 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:35.118292 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8716340a-f1a1-45d4-91d3-a61f85960ddf tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:35.143914 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8716340a-f1a1-45d4-91d3-a61f85960ddf tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:35.145972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8716340a-f1a1-45d4-91d3-a61f85960ddf tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:35.151581 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8716340a-f1a1-45d4-91d3-a61f85960ddf tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.329690 Aug 07 01:13:35.159005 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 228/438] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:34 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:35.168497 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:13:35.608502 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-39ab40f1-765b-4e4d-b952-ec34b6982c31 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:35.627782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-39ab40f1-765b-4e4d-b952-ec34b6982c31 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1318 microversion: 2.1 time: 0.623653 Aug 07 01:13:35.629397 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 211/439] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:35 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1318 bytes in 627 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:36.039249 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8bbce3e3-8cb7-4968-81b8-eaa20d4fab8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:36.039869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8bbce3e3-8cb7-4968-81b8-eaa20d4fab8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:36.047725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8bbce3e3-8cb7-4968-81b8-eaa20d4fab8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:36.049042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8bbce3e3-8cb7-4968-81b8-eaa20d4fab8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:36.049808 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8bbce3e3-8cb7-4968-81b8-eaa20d4fab8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:36.050936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8bbce3e3-8cb7-4968-81b8-eaa20d4fab8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:36.052316 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8bbce3e3-8cb7-4968-81b8-eaa20d4fab8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:36.053104 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8bbce3e3-8cb7-4968-81b8-eaa20d4fab8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:36.187911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-997d25ab-eadd-4551-8b49-414a88d7ba75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:36.190836 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-997d25ab-eadd-4551-8b49-414a88d7ba75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:36.196321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=8 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:13:36.226435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-997d25ab-eadd-4551-8b49-414a88d7ba75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:36.227448 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-997d25ab-eadd-4551-8b49-414a88d7ba75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:36.233042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-997d25ab-eadd-4551-8b49-414a88d7ba75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:36.234105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-997d25ab-eadd-4551-8b49-414a88d7ba75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:36.238720 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-997d25ab-eadd-4551-8b49-414a88d7ba75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:36.239477 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-997d25ab-eadd-4551-8b49-414a88d7ba75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:36.310210 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8bbce3e3-8cb7-4968-81b8-eaa20d4fab8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:36.322193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8bbce3e3-8cb7-4968-81b8-eaa20d4fab8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.287439 Aug 07 01:13:36.323190 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 229/440] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:36 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:36.542457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-997d25ab-eadd-4551-8b49-414a88d7ba75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:36.562078 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-997d25ab-eadd-4551-8b49-414a88d7ba75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:36.565730 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-997d25ab-eadd-4551-8b49-414a88d7ba75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:36.568424 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-997d25ab-eadd-4551-8b49-414a88d7ba75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.387589 Aug 07 01:13:36.570116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 212/441] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:36 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:13:36.655098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fb346507-2279-486f-b3c9-3396c8b74401 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:36.657101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fb346507-2279-486f-b3c9-3396c8b74401 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:36.675242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fb346507-2279-486f-b3c9-3396c8b74401 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:36.676581 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fb346507-2279-486f-b3c9-3396c8b74401 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:36.677637 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fb346507-2279-486f-b3c9-3396c8b74401 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:36.679365 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fb346507-2279-486f-b3c9-3396c8b74401 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:36.680563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fb346507-2279-486f-b3c9-3396c8b74401 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:36.681517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fb346507-2279-486f-b3c9-3396c8b74401 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:36.962858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-fb346507-2279-486f-b3c9-3396c8b74401 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:36.996346 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fb346507-2279-486f-b3c9-3396c8b74401 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1318 microversion: 2.1 time: 0.348280 Aug 07 01:13:36.997393 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 230/442] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:36 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1318 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:37.347785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-40890d76-8f87-4ca8-a234-1c57824e1370 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:37.349117 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-40890d76-8f87-4ca8-a234-1c57824e1370 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:37.355951 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-40890d76-8f87-4ca8-a234-1c57824e1370 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:37.356748 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-40890d76-8f87-4ca8-a234-1c57824e1370 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:37.357428 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-40890d76-8f87-4ca8-a234-1c57824e1370 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:37.358331 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-40890d76-8f87-4ca8-a234-1c57824e1370 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:37.358955 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-40890d76-8f87-4ca8-a234-1c57824e1370 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:37.359546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-40890d76-8f87-4ca8-a234-1c57824e1370 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:37.596537 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4984d567-0477-46ad-ab6e-7028b21084df tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:37.598432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4984d567-0477-46ad-ab6e-7028b21084df tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:37.613133 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4984d567-0477-46ad-ab6e-7028b21084df tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:37.617956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4984d567-0477-46ad-ab6e-7028b21084df tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:37.618829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4984d567-0477-46ad-ab6e-7028b21084df tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:37.621316 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4984d567-0477-46ad-ab6e-7028b21084df tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:37.625174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4984d567-0477-46ad-ab6e-7028b21084df tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:37.625726 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4984d567-0477-46ad-ab6e-7028b21084df tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:37.713944 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-40890d76-8f87-4ca8-a234-1c57824e1370 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:37.725865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-40890d76-8f87-4ca8-a234-1c57824e1370 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.382411 Aug 07 01:13:37.726867 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 213/443] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:37 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:37.890564 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4984d567-0477-46ad-ab6e-7028b21084df tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:37.909140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4984d567-0477-46ad-ab6e-7028b21084df tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:37.913096 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4984d567-0477-46ad-ab6e-7028b21084df tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:37.916987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4984d567-0477-46ad-ab6e-7028b21084df tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 0.325413 Aug 07 01:13:37.919326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 231/444] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:37 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:13:38.023436 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d942ee05-bbb3-4a79-bc27-ec83fb4ffeb2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:38.024433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d942ee05-bbb3-4a79-bc27-ec83fb4ffeb2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:38.034394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d942ee05-bbb3-4a79-bc27-ec83fb4ffeb2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:38.035723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d942ee05-bbb3-4a79-bc27-ec83fb4ffeb2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:38.037234 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d942ee05-bbb3-4a79-bc27-ec83fb4ffeb2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:38.039097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d942ee05-bbb3-4a79-bc27-ec83fb4ffeb2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:38.039800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d942ee05-bbb3-4a79-bc27-ec83fb4ffeb2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:38.041393 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d942ee05-bbb3-4a79-bc27-ec83fb4ffeb2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:38.570402 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d942ee05-bbb3-4a79-bc27-ec83fb4ffeb2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:38.580801 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d942ee05-bbb3-4a79-bc27-ec83fb4ffeb2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1318 microversion: 2.1 time: 0.561346 Aug 07 01:13:38.581773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 214/445] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:38 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1318 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:38.760478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-45387552-30f0-4989-9ad7-23ffbc93dc13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:38.764140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-45387552-30f0-4989-9ad7-23ffbc93dc13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:38.794506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45387552-30f0-4989-9ad7-23ffbc93dc13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:38.795424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45387552-30f0-4989-9ad7-23ffbc93dc13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:38.800307 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45387552-30f0-4989-9ad7-23ffbc93dc13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:38.802019 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45387552-30f0-4989-9ad7-23ffbc93dc13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:38.802769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45387552-30f0-4989-9ad7-23ffbc93dc13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:38.809491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45387552-30f0-4989-9ad7-23ffbc93dc13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:38.948116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1758cb2b-78c9-4c4e-a2e4-1dfa696d1880 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:38.952643 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1758cb2b-78c9-4c4e-a2e4-1dfa696d1880 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:39.003109 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1758cb2b-78c9-4c4e-a2e4-1dfa696d1880 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:39.010437 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1758cb2b-78c9-4c4e-a2e4-1dfa696d1880 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:39.025337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1758cb2b-78c9-4c4e-a2e4-1dfa696d1880 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:39.030967 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1758cb2b-78c9-4c4e-a2e4-1dfa696d1880 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:39.031861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1758cb2b-78c9-4c4e-a2e4-1dfa696d1880 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:39.053406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1758cb2b-78c9-4c4e-a2e4-1dfa696d1880 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:39.260207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-45387552-30f0-4989-9ad7-23ffbc93dc13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:39.288413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-45387552-30f0-4989-9ad7-23ffbc93dc13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.543389 Aug 07 01:13:39.290228 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 232/446] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:38 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:39.611030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2b0a3412-6b09-4dfe-aa28-f85ac09097b1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:39.611802 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2b0a3412-6b09-4dfe-aa28-f85ac09097b1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:39.623485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b0a3412-6b09-4dfe-aa28-f85ac09097b1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:39.627172 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b0a3412-6b09-4dfe-aa28-f85ac09097b1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:39.631441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b0a3412-6b09-4dfe-aa28-f85ac09097b1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:39.632471 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b0a3412-6b09-4dfe-aa28-f85ac09097b1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:39.635642 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b0a3412-6b09-4dfe-aa28-f85ac09097b1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:39.637721 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b0a3412-6b09-4dfe-aa28-f85ac09097b1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:39.952872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2b0a3412-6b09-4dfe-aa28-f85ac09097b1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:39.973030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2b0a3412-6b09-4dfe-aa28-f85ac09097b1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1318 microversion: 2.1 time: 0.365942 Aug 07 01:13:39.993252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 233/447] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:39 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1318 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:40.316581 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dd27f0fd-b9f7-4465-be93-d5278d7aa310 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:40.317210 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dd27f0fd-b9f7-4465-be93-d5278d7aa310 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:40.326611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd27f0fd-b9f7-4465-be93-d5278d7aa310 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:40.328687 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd27f0fd-b9f7-4465-be93-d5278d7aa310 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:40.339111 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd27f0fd-b9f7-4465-be93-d5278d7aa310 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:40.339898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd27f0fd-b9f7-4465-be93-d5278d7aa310 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:40.341977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd27f0fd-b9f7-4465-be93-d5278d7aa310 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:40.343080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd27f0fd-b9f7-4465-be93-d5278d7aa310 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:40.427391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1758cb2b-78c9-4c4e-a2e4-1dfa696d1880 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:40.446777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1758cb2b-78c9-4c4e-a2e4-1dfa696d1880 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:40.448517 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1758cb2b-78c9-4c4e-a2e4-1dfa696d1880 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:40.450132 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1758cb2b-78c9-4c4e-a2e4-1dfa696d1880 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1808 microversion: 2.20 time: 1.509543 Aug 07 01:13:40.451419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 215/448] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:38 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1808 bytes in 1512 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:40.638474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-dd27f0fd-b9f7-4465-be93-d5278d7aa310 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:40.657848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dd27f0fd-b9f7-4465-be93-d5278d7aa310 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.347059 Aug 07 01:13:40.686354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 234/449] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:40 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:41.007622 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0911c024-0813-4591-94d0-2513beaf045a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:41.008483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0911c024-0813-4591-94d0-2513beaf045a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:41.016565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0911c024-0813-4591-94d0-2513beaf045a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:41.017960 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0911c024-0813-4591-94d0-2513beaf045a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:41.018510 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0911c024-0813-4591-94d0-2513beaf045a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:41.019535 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0911c024-0813-4591-94d0-2513beaf045a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:41.020199 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0911c024-0813-4591-94d0-2513beaf045a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:41.020664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0911c024-0813-4591-94d0-2513beaf045a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:41.372650 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0911c024-0813-4591-94d0-2513beaf045a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:41.390944 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0911c024-0813-4591-94d0-2513beaf045a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1318 microversion: 2.1 time: 0.389211 Aug 07 01:13:41.391767 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 216/450] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:41 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1318 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:41.493387 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3ae1943a-2c75-41b4-803d-8b301e9a95fa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:41.495262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3ae1943a-2c75-41b4-803d-8b301e9a95fa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:41.505565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae1943a-2c75-41b4-803d-8b301e9a95fa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:41.506492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae1943a-2c75-41b4-803d-8b301e9a95fa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:41.507130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae1943a-2c75-41b4-803d-8b301e9a95fa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:41.508786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae1943a-2c75-41b4-803d-8b301e9a95fa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:41.509527 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae1943a-2c75-41b4-803d-8b301e9a95fa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:41.509962 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae1943a-2c75-41b4-803d-8b301e9a95fa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:41.644731 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-83f3c39e-3d98-47e3-ae12-629fb6609ecf req-a9293491-771c-4157-94c0-1d615678cae7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc", "name": "network-vif-plugged", "status": "completed", "tag": "3ba58955-f906-4b1b-8224-8a7408f3a52c"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:41.661016 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-83f3c39e-3d98-47e3-ae12-629fb6609ecf req-a9293491-771c-4157-94c0-1d615678cae7 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:41.661505 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-83f3c39e-3d98-47e3-ae12-629fb6609ecf req-a9293491-771c-4157-94c0-1d615678cae7 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:41.661941 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-83f3c39e-3d98-47e3-ae12-629fb6609ecf req-a9293491-771c-4157-94c0-1d615678cae7 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:41.691275 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-83f3c39e-3d98-47e3-ae12-629fb6609ecf req-a9293491-771c-4157-94c0-1d615678cae7 service nova] Creating event network-vif-plugged:3ba58955-f906-4b1b-8224-8a7408f3a52c for instance 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc on hv-cinder-maste Aug 07 01:13:41.699965 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-83f3c39e-3d98-47e3-ae12-629fb6609ecf req-a9293491-771c-4157-94c0-1d615678cae7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059513 Aug 07 01:13:41.701447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 217/451] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:13:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:13:41.715006 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-84496435-9f9f-4920-ae32-3b9895d87e18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:41.716140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-84496435-9f9f-4920-ae32-3b9895d87e18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:41.718861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=67,nova_api:INSERT=3 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:13:41.726384 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84496435-9f9f-4920-ae32-3b9895d87e18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:41.727073 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84496435-9f9f-4920-ae32-3b9895d87e18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:41.727834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84496435-9f9f-4920-ae32-3b9895d87e18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:41.728986 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84496435-9f9f-4920-ae32-3b9895d87e18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:41.729706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84496435-9f9f-4920-ae32-3b9895d87e18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:41.730390 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84496435-9f9f-4920-ae32-3b9895d87e18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:41.780594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3ae1943a-2c75-41b4-803d-8b301e9a95fa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:41.807552 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3ae1943a-2c75-41b4-803d-8b301e9a95fa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:41.811396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3ae1943a-2c75-41b4-803d-8b301e9a95fa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:41.812728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3ae1943a-2c75-41b4-803d-8b301e9a95fa tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1795 microversion: 2.20 time: 0.324625 Aug 07 01:13:41.813542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 235/452] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:13:41 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1795 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:13:42.098288 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-84496435-9f9f-4920-ae32-3b9895d87e18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:42.111192 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-84496435-9f9f-4920-ae32-3b9895d87e18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.407975 Aug 07 01:13:42.112328 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 218/453] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:41 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:42.416904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5bf4adc9-b081-43f4-a805-f8013d34d41c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:42.417593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5bf4adc9-b081-43f4-a805-f8013d34d41c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:42.428401 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bf4adc9-b081-43f4-a805-f8013d34d41c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:42.428830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bf4adc9-b081-43f4-a805-f8013d34d41c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:42.429636 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bf4adc9-b081-43f4-a805-f8013d34d41c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:42.430782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bf4adc9-b081-43f4-a805-f8013d34d41c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:42.431722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bf4adc9-b081-43f4-a805-f8013d34d41c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:42.432834 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bf4adc9-b081-43f4-a805-f8013d34d41c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:42.747702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5bf4adc9-b081-43f4-a805-f8013d34d41c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:42.759590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5bf4adc9-b081-43f4-a805-f8013d34d41c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1318 microversion: 2.1 time: 0.350508 Aug 07 01:13:42.760890 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 236/454] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:42 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1318 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:43.143518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-86a57ca6-ceb0-48b3-9161-f64006c7518f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:43.145794 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-86a57ca6-ceb0-48b3-9161-f64006c7518f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:43.154203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86a57ca6-ceb0-48b3-9161-f64006c7518f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:43.154987 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86a57ca6-ceb0-48b3-9161-f64006c7518f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:43.155737 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86a57ca6-ceb0-48b3-9161-f64006c7518f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:43.157060 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86a57ca6-ceb0-48b3-9161-f64006c7518f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:43.158444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86a57ca6-ceb0-48b3-9161-f64006c7518f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:43.159076 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86a57ca6-ceb0-48b3-9161-f64006c7518f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:43.392068 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-86a57ca6-ceb0-48b3-9161-f64006c7518f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:43.406925 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-86a57ca6-ceb0-48b3-9161-f64006c7518f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.268641 Aug 07 01:13:43.408807 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 219/455] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:43 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:43.801099 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bea553ac-a58f-4538-a8c2-8e168dae61ed tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:43.801925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bea553ac-a58f-4538-a8c2-8e168dae61ed tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:43.809536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bea553ac-a58f-4538-a8c2-8e168dae61ed tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:43.810417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bea553ac-a58f-4538-a8c2-8e168dae61ed tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:43.811245 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bea553ac-a58f-4538-a8c2-8e168dae61ed tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:43.812076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bea553ac-a58f-4538-a8c2-8e168dae61ed tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:43.812901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bea553ac-a58f-4538-a8c2-8e168dae61ed tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:43.813546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bea553ac-a58f-4538-a8c2-8e168dae61ed tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:44.060441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-bea553ac-a58f-4538-a8c2-8e168dae61ed tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:44.083952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bea553ac-a58f-4538-a8c2-8e168dae61ed tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1318 microversion: 2.1 time: 0.287317 Aug 07 01:13:44.085504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 237/456] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:43 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1318 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:44.135518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fb241d15-4f56-40c9-9a60-c8d04099af3d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Action: 'create', calling method: .version_select at 0x7f1fad6e8310>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1022328377", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJo97LDXHZaFhuKl6lowrSE7WRTqzXyF8aC7N3/tYvuLuenMk6PGE++njqowTi0r80hgGkb+6gu5bZgr6XvvezQGwCpZ/RnCGx3GrBxycTLRwcT7/nSQy/AsDIm0Fvnlwg=="}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:44.153601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-fb241d15-4f56-40c9-9a60-c8d04099af3d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Getting quotas for project b73b98c606a34d1faf0a31a9f6c59e56. Resources: {'key_pairs'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:13:44.160755 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-fb241d15-4f56-40c9-9a60-c8d04099af3d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Getting quotas for user 642126380bcf4446bf1e320f93139034 and project b73b98c606a34d1faf0a31a9f6c59e56. Resources: {'key_pairs'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:13:44.191670 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fb241d15-4f56-40c9-9a60-c8d04099af3d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.329520 Aug 07 01:13:44.199353 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 220/457] 10.222.0.26 () {60 vars in 1159 bytes} [Sun Aug 7 01:13:43 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:13:44.450889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d764b08f-65cb-45bf-b348-4b8d145edadb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:44.451996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d764b08f-65cb-45bf-b348-4b8d145edadb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:44.462889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d764b08f-65cb-45bf-b348-4b8d145edadb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:44.463828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d764b08f-65cb-45bf-b348-4b8d145edadb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:44.464635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d764b08f-65cb-45bf-b348-4b8d145edadb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:44.465979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d764b08f-65cb-45bf-b348-4b8d145edadb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:44.467590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d764b08f-65cb-45bf-b348-4b8d145edadb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:44.468753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d764b08f-65cb-45bf-b348-4b8d145edadb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:44.692356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d764b08f-65cb-45bf-b348-4b8d145edadb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:44.703158 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d764b08f-65cb-45bf-b348-4b8d145edadb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.260168 Aug 07 01:13:44.704050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 238/458] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:44 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:45.114133 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-29bc7e59-e2cb-431e-b9a7-5e190b21bfda tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:45.115420 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-29bc7e59-e2cb-431e-b9a7-5e190b21bfda tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:45.127693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-29bc7e59-e2cb-431e-b9a7-5e190b21bfda tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:45.128739 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-29bc7e59-e2cb-431e-b9a7-5e190b21bfda tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:45.129587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-29bc7e59-e2cb-431e-b9a7-5e190b21bfda tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:45.130980 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-29bc7e59-e2cb-431e-b9a7-5e190b21bfda tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:45.131904 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-29bc7e59-e2cb-431e-b9a7-5e190b21bfda tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:45.133313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-29bc7e59-e2cb-431e-b9a7-5e190b21bfda tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:45.412786 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-29bc7e59-e2cb-431e-b9a7-5e190b21bfda tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:45.421590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-29bc7e59-e2cb-431e-b9a7-5e190b21bfda tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1507 microversion: 2.1 time: 0.317119 Aug 07 01:13:45.422609 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 221/459] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:45 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1507 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:45.731126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b935a086-e877-4959-b1bb-147df42024f6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:45.731920 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b935a086-e877-4959-b1bb-147df42024f6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:45.748374 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b935a086-e877-4959-b1bb-147df42024f6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:45.748942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b935a086-e877-4959-b1bb-147df42024f6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:45.749542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b935a086-e877-4959-b1bb-147df42024f6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:45.755273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b935a086-e877-4959-b1bb-147df42024f6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:45.755816 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b935a086-e877-4959-b1bb-147df42024f6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:45.757116 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b935a086-e877-4959-b1bb-147df42024f6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:45.991419 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b935a086-e877-4959-b1bb-147df42024f6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:46.007304 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b935a086-e877-4959-b1bb-147df42024f6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.281429 Aug 07 01:13:46.008582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 239/460] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:45 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:47.046270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-50370da2-e8e9-4d9d-82b9-e7a9f45d31cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:47.047194 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-50370da2-e8e9-4d9d-82b9-e7a9f45d31cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:47.056534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-50370da2-e8e9-4d9d-82b9-e7a9f45d31cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:47.057038 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-50370da2-e8e9-4d9d-82b9-e7a9f45d31cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:47.057500 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-50370da2-e8e9-4d9d-82b9-e7a9f45d31cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:47.058253 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-50370da2-e8e9-4d9d-82b9-e7a9f45d31cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:47.058730 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-50370da2-e8e9-4d9d-82b9-e7a9f45d31cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:47.059338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-50370da2-e8e9-4d9d-82b9-e7a9f45d31cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:47.082454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:13:47.105369 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-42de4a79-49d8-4993-8168-7e30bea061c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:47.106979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-42de4a79-49d8-4993-8168-7e30bea061c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:47.116865 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-42de4a79-49d8-4993-8168-7e30bea061c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:47.117397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-42de4a79-49d8-4993-8168-7e30bea061c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:47.117878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-42de4a79-49d8-4993-8168-7e30bea061c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:47.118751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-42de4a79-49d8-4993-8168-7e30bea061c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:47.119353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-42de4a79-49d8-4993-8168-7e30bea061c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:47.119891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-42de4a79-49d8-4993-8168-7e30bea061c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:47.287255 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-50370da2-e8e9-4d9d-82b9-e7a9f45d31cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:47.299638 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-50370da2-e8e9-4d9d-82b9-e7a9f45d31cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.258979 Aug 07 01:13:47.300256 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 222/461] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:47 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:47.352635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.objects.instance [None req-42de4a79-49d8-4993-8168-7e30bea061c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lazy-loading 'services' on Instance uuid 4f3f0774-3a34-4f19-9233-c4e55c7f5903 {{(pid=117983) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:13:47.596045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:13:47.615210 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-42de4a79-49d8-4993-8168-7e30bea061c6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "DELETE /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/os-volume_attachments/c045b917-ed68-46a0-a932-c7b3fe80957f" status: 202 len: 0 microversion: 2.1 time: 0.513521 Aug 07 01:13:47.619506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 240/462] 10.222.0.26 () {58 vars in 1416 bytes} [Sun Aug 7 01:13:47 2022] DELETE /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/os-volume_attachments/c045b917-ed68-46a0-a932-c7b3fe80957f => generated 0 bytes in 516 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:13:48.323467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-54ee1bb3-f1b0-4dd8-9c71-5e9456613337 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:48.324397 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-54ee1bb3-f1b0-4dd8-9c71-5e9456613337 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:48.334146 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-54ee1bb3-f1b0-4dd8-9c71-5e9456613337 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:48.334797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-54ee1bb3-f1b0-4dd8-9c71-5e9456613337 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:48.335576 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-54ee1bb3-f1b0-4dd8-9c71-5e9456613337 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:48.337193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-54ee1bb3-f1b0-4dd8-9c71-5e9456613337 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:48.338001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-54ee1bb3-f1b0-4dd8-9c71-5e9456613337 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:48.339052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-54ee1bb3-f1b0-4dd8-9c71-5e9456613337 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:48.526371 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-54ee1bb3-f1b0-4dd8-9c71-5e9456613337 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:48.539226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-54ee1bb3-f1b0-4dd8-9c71-5e9456613337 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.219540 Aug 07 01:13:48.540306 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 223/463] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:48 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:49.563363 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ed47514f-04d8-4c9c-88c7-7d0544e12e71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:49.577189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ed47514f-04d8-4c9c-88c7-7d0544e12e71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:49.577189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed47514f-04d8-4c9c-88c7-7d0544e12e71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:49.577189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed47514f-04d8-4c9c-88c7-7d0544e12e71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:49.577189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed47514f-04d8-4c9c-88c7-7d0544e12e71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:49.577931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed47514f-04d8-4c9c-88c7-7d0544e12e71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:49.577931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed47514f-04d8-4c9c-88c7-7d0544e12e71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:49.577931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed47514f-04d8-4c9c-88c7-7d0544e12e71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:49.764909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ed47514f-04d8-4c9c-88c7-7d0544e12e71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:49.774711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ed47514f-04d8-4c9c-88c7-7d0544e12e71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.215717 Aug 07 01:13:49.775952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 241/464] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:49 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:49.881933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b20d6d08-6313-461d-9ddb-3999d39209b3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:49.889876 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b20d6d08-6313-461d-9ddb-3999d39209b3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:49.890345 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b20d6d08-6313-461d-9ddb-3999d39209b3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:49.890789 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b20d6d08-6313-461d-9ddb-3999d39209b3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:49.916422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b20d6d08-6313-461d-9ddb-3999d39209b3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Going to try to stop instance {{(pid=117984) force_stop /opt/stack/nova/nova/compute/api.py:2709}} Aug 07 01:13:49.992402 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b20d6d08-6313-461d-9ddb-3999d39209b3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "POST /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/action" status: 202 len: 0 microversion: 2.1 time: 0.114705 Aug 07 01:13:49.993494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 224/465] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:13:49 2022] POST /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/action => generated 0 bytes in 118 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:13:50.007256 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-39c2bd99-e907-40b4-ab50-5e32346dca8c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:50.010391 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-39c2bd99-e907-40b4-ab50-5e32346dca8c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:50.014455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=64 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:13:50.024656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-39c2bd99-e907-40b4-ab50-5e32346dca8c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:50.026006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-39c2bd99-e907-40b4-ab50-5e32346dca8c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:50.026182 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-39c2bd99-e907-40b4-ab50-5e32346dca8c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:50.027156 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-39c2bd99-e907-40b4-ab50-5e32346dca8c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:50.027992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-39c2bd99-e907-40b4-ab50-5e32346dca8c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:50.028981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-39c2bd99-e907-40b4-ab50-5e32346dca8c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:50.242202 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-39c2bd99-e907-40b4-ab50-5e32346dca8c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:50.252008 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-39c2bd99-e907-40b4-ab50-5e32346dca8c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1678 microversion: 2.1 time: 0.251429 Aug 07 01:13:50.252638 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 242/466] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:49 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1678 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:50.811029 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8f189298-2de0-47c5-aeff-a6096e99b91a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:50.811904 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8f189298-2de0-47c5-aeff-a6096e99b91a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:50.820186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f189298-2de0-47c5-aeff-a6096e99b91a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:50.820716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f189298-2de0-47c5-aeff-a6096e99b91a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:50.821424 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f189298-2de0-47c5-aeff-a6096e99b91a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:50.822154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f189298-2de0-47c5-aeff-a6096e99b91a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:50.822613 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f189298-2de0-47c5-aeff-a6096e99b91a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:50.823052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f189298-2de0-47c5-aeff-a6096e99b91a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:51.021704 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8f189298-2de0-47c5-aeff-a6096e99b91a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:51.034518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8f189298-2de0-47c5-aeff-a6096e99b91a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1503 microversion: 2.1 time: 0.227810 Aug 07 01:13:51.035418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 225/467] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:50 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1503 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:51.056509 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9294cee1-e6b5-483a-b251-a96b891133fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "8f875f1c-f74e-4180-98e8-0cdda1a70d95"}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:51.059053 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9294cee1-e6b5-483a-b251-a96b891133fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:51.066653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9294cee1-e6b5-483a-b251-a96b891133fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:51.067674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9294cee1-e6b5-483a-b251-a96b891133fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:51.067674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9294cee1-e6b5-483a-b251-a96b891133fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:51.069028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9294cee1-e6b5-483a-b251-a96b891133fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:51.069856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9294cee1-e6b5-483a-b251-a96b891133fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:51.070630 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9294cee1-e6b5-483a-b251-a96b891133fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:51.109623 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-9294cee1-e6b5-483a-b251-a96b891133fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] HTTP exception thrown: Cannot 'attach_volume' instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 while it is in vm_state rescued Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9294cee1-e6b5-483a-b251-a96b891133fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Request method failure captured: Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: POST /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530/os-volume_attachments HTTP/1.1 Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 74 Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xFan_-oNUM4AqSt7Wni9p606OoWcdJLz4SIr5VHLwgODJGL0BBSKB64xI7IBMYJhEzGuAdjPCyw6B3DmJ9opPnSTNGpjpiLZ8MHcy7T0m2nWHFImY2c23r73oTOGBUNFDYYRljAnrO4awdzCByqv2_7MGWKYhYTsQ-EtnOqUmEQZ_Q Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 9badaa18dcf64810b10dd7bf03d8e40d Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1786214574 Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:13:51.113198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:13:51.115908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:13:51.115908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1786214574 Aug 07 01:13:51.115908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 9badaa18dcf64810b10dd7bf03d8e40d Aug 07 01:13:51.115908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1786214574 Aug 07 01:13:51.115908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-ServerRescueNegativeTestJSON-1786214574-project Aug 07 01:13:51.115908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:13:51.115908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:13:51.115908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: ec931c9cc56a462991413926d34dfa46 Aug 07 01:13:51.115908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1786214574-project Aug 07 01:13:51.115908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Aug 07 01:13:51.115908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {"volumeAttachment": {"volumeId": "8f875f1c-f74e-4180-98e8-0cdda1a70d95"}} Aug 07 01:13:51.115908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Cannot 'attach_volume' instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 while it is in vm_state rescued Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'server_id': '99e925dd-a3a5-44ab-bf20-9c80807d5530', 'body': {'volumeAttachment': {'volumeId': '8f875f1c-f74e-4180-98e8-0cdda1a70d95'}}} Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 in vm_state rescued. Cannot attach_volume while the instance is in this state. Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:13:51.117366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 while it is in vm_state rescued Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:13:51.119944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:13:51.122543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:13:51.122543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:13:51.122543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 while it is in vm_state rescued Aug 07 01:13:51.122543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:13:51.122543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9294cee1-e6b5-483a-b251-a96b891133fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Returning 409 to user: Cannot 'attach_volume' instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 while it is in vm_state rescued {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:13:51.122543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9294cee1-e6b5-483a-b251-a96b891133fa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "POST /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.063316 Aug 07 01:13:51.122543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 243/468] 10.222.0.26 () {60 vars in 1323 bytes} [Sun Aug 7 01:13:51 2022] POST /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530/os-volume_attachments => generated 152 bytes in 65 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Aug 07 01:13:51.129499 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1b5c606b-441c-411a-aa19-8096b54758db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:51.132736 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1b5c606b-441c-411a-aa19-8096b54758db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:51.142919 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5c606b-441c-411a-aa19-8096b54758db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:51.143422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5c606b-441c-411a-aa19-8096b54758db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:51.143873 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5c606b-441c-411a-aa19-8096b54758db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:51.144648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5c606b-441c-411a-aa19-8096b54758db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:51.145115 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5c606b-441c-411a-aa19-8096b54758db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:51.145548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5c606b-441c-411a-aa19-8096b54758db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:51.226026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1b5c606b-441c-411a-aa19-8096b54758db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "POST /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530/action" status: 202 len: 0 microversion: 2.1 time: 0.101656 Aug 07 01:13:51.227199 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 226/469] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:13:51 2022] POST /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530/action => generated 0 bytes in 104 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:13:51.236908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7b61ca05-5856-4a8f-ad5c-43c948ed343a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:51.237727 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7b61ca05-5856-4a8f-ad5c-43c948ed343a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:51.246519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b61ca05-5856-4a8f-ad5c-43c948ed343a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:51.247140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b61ca05-5856-4a8f-ad5c-43c948ed343a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:51.247739 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b61ca05-5856-4a8f-ad5c-43c948ed343a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:51.248537 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b61ca05-5856-4a8f-ad5c-43c948ed343a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:51.249164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b61ca05-5856-4a8f-ad5c-43c948ed343a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:51.249456 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b61ca05-5856-4a8f-ad5c-43c948ed343a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:51.278893 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4e042338-122c-4f90-8a47-b878389e3bda tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:51.280148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4e042338-122c-4f90-8a47-b878389e3bda tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:51.293790 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4e042338-122c-4f90-8a47-b878389e3bda tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:51.294797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4e042338-122c-4f90-8a47-b878389e3bda tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:51.295709 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4e042338-122c-4f90-8a47-b878389e3bda tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:51.297337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4e042338-122c-4f90-8a47-b878389e3bda tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:51.299762 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4e042338-122c-4f90-8a47-b878389e3bda tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:51.300968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4e042338-122c-4f90-8a47-b878389e3bda tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:51.478462 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7b61ca05-5856-4a8f-ad5c-43c948ed343a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:51.494213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7b61ca05-5856-4a8f-ad5c-43c948ed343a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1511 microversion: 2.1 time: 0.261934 Aug 07 01:13:51.495771 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 244/470] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:51 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1511 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:51.720411 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4e042338-122c-4f90-8a47-b878389e3bda tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:51.733145 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4e042338-122c-4f90-8a47-b878389e3bda tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1678 microversion: 2.1 time: 0.461168 Aug 07 01:13:51.744434 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 227/471] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:51 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1678 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:52.197979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c5386593-66fb-42eb-9a61-d38a7047720c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:52.198843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c5386593-66fb-42eb-9a61-d38a7047720c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:52.206040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c5386593-66fb-42eb-9a61-d38a7047720c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:52.206540 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c5386593-66fb-42eb-9a61-d38a7047720c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:52.206989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c5386593-66fb-42eb-9a61-d38a7047720c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:52.207728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c5386593-66fb-42eb-9a61-d38a7047720c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:52.208190 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c5386593-66fb-42eb-9a61-d38a7047720c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:52.208661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c5386593-66fb-42eb-9a61-d38a7047720c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:52.241205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c5386593-66fb-42eb-9a61-d38a7047720c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc/ips" status: 200 len: 151 microversion: 2.1 time: 0.047487 Aug 07 01:13:52.241799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 245/472] 10.222.0.26 () {58 vars in 1248 bytes} [Sun Aug 7 01:13:52 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc/ips => generated 151 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:13:52.265911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-91cd969e-3f2f-497e-9a37-1c8440f85629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "fa48cc70-a60b-440f-8b1a-09ffff661711"}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:52.268520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-91cd969e-3f2f-497e-9a37-1c8440f85629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:52.280644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-91cd969e-3f2f-497e-9a37-1c8440f85629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:52.281192 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-91cd969e-3f2f-497e-9a37-1c8440f85629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:52.281869 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-91cd969e-3f2f-497e-9a37-1c8440f85629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:52.285736 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-91cd969e-3f2f-497e-9a37-1c8440f85629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:52.286477 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-91cd969e-3f2f-497e-9a37-1c8440f85629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:52.287349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-91cd969e-3f2f-497e-9a37-1c8440f85629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:52.417301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-91cd969e-3f2f-497e-9a37-1c8440f85629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] HTTP exception thrown: Invalid volume: volume fa48cc70-a60b-440f-8b1a-09ffff661711 is already attached to instances: 96f9a9ce-f29e-48be-a36b-8518126a09ee Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-91cd969e-3f2f-497e-9a37-1c8440f85629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Request method failure captured: Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: POST /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc/os-volume_attachments HTTP/1.1 Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 74 Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xD021Vt2D0N359UOKX9xymFYpnZ79FQqLbXwvg5UbTRYuTTlR_HceEoHaOg5fvzy7nSB6FZNa7SVaU3q4lxIRVTXMupRK2mSDBaPiXGNUdpidSOmvxEoLqM4Zox-kqkoV9_K0_oxsudsRHb6jwecdh1rLe0262HFBUWOJmPZoVity8 Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:13:52.419422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:13:52.422137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:13:52.422137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:13:52.422137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:13:52.422137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:13:52.422137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:13:52.422137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:13:52.422137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:13:52.422137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: e65f0f1ba54f483abff788974b1256cb Aug 07 01:13:52.422137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:13:52.422137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Aug 07 01:13:52.422137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {"volumeAttachment": {"volumeId": "fa48cc70-a60b-440f-8b1a-09ffff661711"}} Aug 07 01:13:52.422137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Invalid volume: volume fa48cc70-a60b-440f-8b1a-09ffff661711 is already attached to instances: 96f9a9ce-f29e-48be-a36b-8518126a09ee Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'server_id': '54a16243-ecd7-4152-a1d3-bf92cc3709cc', 'body': {'volumeAttachment': {'volumeId': 'fa48cc70-a60b-440f-8b1a-09ffff661711'}}} Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5066, in attach_volume Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4906, in _check_volume_already_attached Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume fa48cc70-a60b-440f-8b1a-09ffff661711 is already attached to instances: 96f9a9ce-f29e-48be-a36b-8518126a09ee Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:13:52.424300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume fa48cc70-a60b-440f-8b1a-09ffff661711 is already attached to instances: 96f9a9ce-f29e-48be-a36b-8518126a09ee Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:13:52.426724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:13:52.429355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:13:52.429355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:13:52.429355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:13:52.429355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume fa48cc70-a60b-440f-8b1a-09ffff661711 is already attached to instances: 96f9a9ce-f29e-48be-a36b-8518126a09ee Aug 07 01:13:52.429355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:13:52.429355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-91cd969e-3f2f-497e-9a37-1c8440f85629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Returning 400 to user: Invalid volume: volume fa48cc70-a60b-440f-8b1a-09ffff661711 is already attached to instances: 96f9a9ce-f29e-48be-a36b-8518126a09ee {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:13:52.429355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-91cd969e-3f2f-497e-9a37-1c8440f85629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "POST /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.159444 Aug 07 01:13:52.429355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 228/473] 10.222.0.26 () {60 vars in 1323 bytes} [Sun Aug 7 01:13:52 2022] POST /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc/os-volume_attachments => generated 174 bytes in 161 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 07 01:13:52.435661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e0d74014-e423-448d-ae64-95c456ca73f2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:52.436603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e0d74014-e423-448d-ae64-95c456ca73f2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:52.447659 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0d74014-e423-448d-ae64-95c456ca73f2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:52.448580 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0d74014-e423-448d-ae64-95c456ca73f2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:52.449360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0d74014-e423-448d-ae64-95c456ca73f2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:52.450650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0d74014-e423-448d-ae64-95c456ca73f2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:52.451464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0d74014-e423-448d-ae64-95c456ca73f2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:52.452197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0d74014-e423-448d-ae64-95c456ca73f2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:52.494374 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e0d74014-e423-448d-ae64-95c456ca73f2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Going to try to terminate instance {{(pid=117983) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:13:52.530170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1445cf8b-1dd5-48e0-8b80-f84dbec28a55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:52.530875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1445cf8b-1dd5-48e0-8b80-f84dbec28a55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:52.539623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1445cf8b-1dd5-48e0-8b80-f84dbec28a55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:52.540473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1445cf8b-1dd5-48e0-8b80-f84dbec28a55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:52.541131 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1445cf8b-1dd5-48e0-8b80-f84dbec28a55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:52.542283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1445cf8b-1dd5-48e0-8b80-f84dbec28a55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:52.542967 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1445cf8b-1dd5-48e0-8b80-f84dbec28a55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:52.543413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1445cf8b-1dd5-48e0-8b80-f84dbec28a55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:52.617317 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e0d74014-e423-448d-ae64-95c456ca73f2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "DELETE /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 204 len: 0 microversion: 2.1 time: 0.188971 Aug 07 01:13:52.618343 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 246/474] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:13:52 2022] DELETE /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 07 01:13:52.631255 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-25bfa857-4f3b-4ef7-af0b-e739eb540166 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:52.632271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-25bfa857-4f3b-4ef7-af0b-e739eb540166 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:52.648080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25bfa857-4f3b-4ef7-af0b-e739eb540166 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:52.649211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25bfa857-4f3b-4ef7-af0b-e739eb540166 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:52.649886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25bfa857-4f3b-4ef7-af0b-e739eb540166 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:52.655617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25bfa857-4f3b-4ef7-af0b-e739eb540166 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:52.656470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25bfa857-4f3b-4ef7-af0b-e739eb540166 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:52.657158 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25bfa857-4f3b-4ef7-af0b-e739eb540166 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:52.844441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1445cf8b-1dd5-48e0-8b80-f84dbec28a55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:52.858317 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1445cf8b-1dd5-48e0-8b80-f84dbec28a55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1511 microversion: 2.1 time: 0.332623 Aug 07 01:13:52.872519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 229/475] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:52 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1511 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:52.872519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e6a6c966-62e5-4916-8d37-e59bdd376687 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:52.872519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e6a6c966-62e5-4916-8d37-e59bdd376687 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:52.872932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6a6c966-62e5-4916-8d37-e59bdd376687 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:52.873234 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6a6c966-62e5-4916-8d37-e59bdd376687 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:52.873715 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6a6c966-62e5-4916-8d37-e59bdd376687 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:52.877519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6a6c966-62e5-4916-8d37-e59bdd376687 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:52.877519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6a6c966-62e5-4916-8d37-e59bdd376687 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:52.877519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6a6c966-62e5-4916-8d37-e59bdd376687 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:53.005051 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-25bfa857-4f3b-4ef7-af0b-e739eb540166 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:53.012555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-25bfa857-4f3b-4ef7-af0b-e739eb540166 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1684 microversion: 2.1 time: 0.388453 Aug 07 01:13:53.013249 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 247/476] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:52 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1684 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:53.075928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e6a6c966-62e5-4916-8d37-e59bdd376687 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:53.087614 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e6a6c966-62e5-4916-8d37-e59bdd376687 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1678 microversion: 2.1 time: 0.227481 Aug 07 01:13:53.088473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 230/477] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:52 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1678 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:53.893146 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7e3e64dd-4fe8-4942-961b-4dd3263de483 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:53.893925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7e3e64dd-4fe8-4942-961b-4dd3263de483 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:53.901050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e3e64dd-4fe8-4942-961b-4dd3263de483 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:53.901548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e3e64dd-4fe8-4942-961b-4dd3263de483 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:53.901990 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e3e64dd-4fe8-4942-961b-4dd3263de483 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:53.902702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e3e64dd-4fe8-4942-961b-4dd3263de483 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:53.903162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e3e64dd-4fe8-4942-961b-4dd3263de483 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:53.903592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e3e64dd-4fe8-4942-961b-4dd3263de483 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:54.047761 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-281a64c7-af26-4c24-aee6-14501c7089dd tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:54.048910 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-281a64c7-af26-4c24-aee6-14501c7089dd tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:54.055207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7e3e64dd-4fe8-4942-961b-4dd3263de483 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:54.055691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-281a64c7-af26-4c24-aee6-14501c7089dd tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:54.056215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-281a64c7-af26-4c24-aee6-14501c7089dd tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:54.056685 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-281a64c7-af26-4c24-aee6-14501c7089dd tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:54.057493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-281a64c7-af26-4c24-aee6-14501c7089dd tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:54.057853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-281a64c7-af26-4c24-aee6-14501c7089dd tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:54.058278 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-281a64c7-af26-4c24-aee6-14501c7089dd tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:54.063598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7e3e64dd-4fe8-4942-961b-4dd3263de483 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1511 microversion: 2.1 time: 0.175346 Aug 07 01:13:54.065003 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 248/478] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:53 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1511 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:54.110342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-80d8a052-a177-4a41-baf4-364a59b88d00 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:54.111899 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-80d8a052-a177-4a41-baf4-364a59b88d00 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:54.122595 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-80d8a052-a177-4a41-baf4-364a59b88d00 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:54.123227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-80d8a052-a177-4a41-baf4-364a59b88d00 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:54.123703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-80d8a052-a177-4a41-baf4-364a59b88d00 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:54.124979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-80d8a052-a177-4a41-baf4-364a59b88d00 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:54.124979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-80d8a052-a177-4a41-baf4-364a59b88d00 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:54.125386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-80d8a052-a177-4a41-baf4-364a59b88d00 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:54.171160 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=4 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:13:54.226190 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-281a64c7-af26-4c24-aee6-14501c7089dd tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:54.236999 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-281a64c7-af26-4c24-aee6-14501c7089dd tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1684 microversion: 2.1 time: 0.193071 Aug 07 01:13:54.237601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 231/479] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:54 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1684 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:54.331536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-80d8a052-a177-4a41-baf4-364a59b88d00 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:54.339874 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-80d8a052-a177-4a41-baf4-364a59b88d00 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1678 microversion: 2.1 time: 0.233371 Aug 07 01:13:54.340526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 249/480] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:54 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1678 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:55.088698 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b3a27864-17f1-466f-8acf-c5ea035b8939 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:55.089336 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b3a27864-17f1-466f-8acf-c5ea035b8939 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:55.096571 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b3a27864-17f1-466f-8acf-c5ea035b8939 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:55.097067 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b3a27864-17f1-466f-8acf-c5ea035b8939 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:55.097502 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b3a27864-17f1-466f-8acf-c5ea035b8939 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:55.098224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b3a27864-17f1-466f-8acf-c5ea035b8939 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:55.098693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b3a27864-17f1-466f-8acf-c5ea035b8939 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:55.099102 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b3a27864-17f1-466f-8acf-c5ea035b8939 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:55.255131 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b3a27864-17f1-466f-8acf-c5ea035b8939 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:55.259334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4062a248-823d-4a0d-b4ec-225c68e98f06 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:55.260087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4062a248-823d-4a0d-b4ec-225c68e98f06 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:55.265293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b3a27864-17f1-466f-8acf-c5ea035b8939 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1511 microversion: 2.1 time: 0.179451 Aug 07 01:13:55.268450 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4062a248-823d-4a0d-b4ec-225c68e98f06 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:55.269197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4062a248-823d-4a0d-b4ec-225c68e98f06 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:55.270655 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 232/481] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:55 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1511 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:55.271196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4062a248-823d-4a0d-b4ec-225c68e98f06 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:55.271936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4062a248-823d-4a0d-b4ec-225c68e98f06 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:55.272743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4062a248-823d-4a0d-b4ec-225c68e98f06 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:55.273243 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4062a248-823d-4a0d-b4ec-225c68e98f06 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:55.362735 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2c506ef3-e844-4bd6-8042-ac42a5e0dfd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:55.363356 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2c506ef3-e844-4bd6-8042-ac42a5e0dfd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:55.369719 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2c506ef3-e844-4bd6-8042-ac42a5e0dfd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:55.370196 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2c506ef3-e844-4bd6-8042-ac42a5e0dfd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:55.370638 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2c506ef3-e844-4bd6-8042-ac42a5e0dfd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:55.371363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2c506ef3-e844-4bd6-8042-ac42a5e0dfd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:55.371824 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2c506ef3-e844-4bd6-8042-ac42a5e0dfd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:55.372270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2c506ef3-e844-4bd6-8042-ac42a5e0dfd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:55.460604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4062a248-823d-4a0d-b4ec-225c68e98f06 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:55.472085 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4062a248-823d-4a0d-b4ec-225c68e98f06 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1684 microversion: 2.1 time: 0.216744 Aug 07 01:13:55.473253 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 250/482] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:55 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1684 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:55.529850 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2c506ef3-e844-4bd6-8042-ac42a5e0dfd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:55.539979 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2c506ef3-e844-4bd6-8042-ac42a5e0dfd1 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1678 microversion: 2.1 time: 0.182005 Aug 07 01:13:55.540797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 233/483] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:55 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1678 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:56.298892 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3ac227d5-3404-485e-b609-ef8babe3cce4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:56.300058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3ac227d5-3404-485e-b609-ef8babe3cce4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:56.311704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ac227d5-3404-485e-b609-ef8babe3cce4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:56.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ac227d5-3404-485e-b609-ef8babe3cce4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:56.312876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ac227d5-3404-485e-b609-ef8babe3cce4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:56.313642 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ac227d5-3404-485e-b609-ef8babe3cce4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:56.314759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ac227d5-3404-485e-b609-ef8babe3cce4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:56.315296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ac227d5-3404-485e-b609-ef8babe3cce4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:56.500314 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0d835027-8fb9-441d-ab60-91d6e9d76a4c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:56.502219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0d835027-8fb9-441d-ab60-91d6e9d76a4c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:56.514800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d835027-8fb9-441d-ab60-91d6e9d76a4c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:56.515706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d835027-8fb9-441d-ab60-91d6e9d76a4c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:56.516578 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d835027-8fb9-441d-ab60-91d6e9d76a4c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:56.517944 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d835027-8fb9-441d-ab60-91d6e9d76a4c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:56.518801 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d835027-8fb9-441d-ab60-91d6e9d76a4c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:56.519586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d835027-8fb9-441d-ab60-91d6e9d76a4c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:56.730360 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0d835027-8fb9-441d-ab60-91d6e9d76a4c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:56.742994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3ac227d5-3404-485e-b609-ef8babe3cce4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:56.745287 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0d835027-8fb9-441d-ab60-91d6e9d76a4c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1684 microversion: 2.1 time: 0.252459 Aug 07 01:13:56.746245 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 234/484] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:56 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1684 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:56.754655 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b580c661-1950-45d9-aba7-a2c559704435 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:56.755635 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b580c661-1950-45d9-aba7-a2c559704435 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:56.768047 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b580c661-1950-45d9-aba7-a2c559704435 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:56.769158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b580c661-1950-45d9-aba7-a2c559704435 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:56.769929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b580c661-1950-45d9-aba7-a2c559704435 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:56.771776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b580c661-1950-45d9-aba7-a2c559704435 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:56.772786 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b580c661-1950-45d9-aba7-a2c559704435 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:56.773586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b580c661-1950-45d9-aba7-a2c559704435 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:56.775787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3ac227d5-3404-485e-b609-ef8babe3cce4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1517 microversion: 2.1 time: 0.484413 Aug 07 01:13:56.777631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 251/485] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:56 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1517 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:57.010164 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b580c661-1950-45d9-aba7-a2c559704435 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:57.023891 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b580c661-1950-45d9-aba7-a2c559704435 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1678 microversion: 2.1 time: 0.276131 Aug 07 01:13:57.024834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 235/486] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:56 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1678 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:57.777822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-68a195bf-9c16-4c81-989e-1012b361147d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:57.781496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-68a195bf-9c16-4c81-989e-1012b361147d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:57.796840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68a195bf-9c16-4c81-989e-1012b361147d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:57.797782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68a195bf-9c16-4c81-989e-1012b361147d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:57.799072 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68a195bf-9c16-4c81-989e-1012b361147d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:57.800436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68a195bf-9c16-4c81-989e-1012b361147d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:57.801292 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68a195bf-9c16-4c81-989e-1012b361147d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:57.802097 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68a195bf-9c16-4c81-989e-1012b361147d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:57.947226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-68a195bf-9c16-4c81-989e-1012b361147d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:57.969808 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-68a195bf-9c16-4c81-989e-1012b361147d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 200 len: 1616 microversion: 2.1 time: 0.199084 Aug 07 01:13:57.969808 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 252/487] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:57 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 1616 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:13:58.053708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-215e8184-d544-4f71-a758-c8b409f9d9bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:58.054745 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-215e8184-d544-4f71-a758-c8b409f9d9bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:58.065761 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-215e8184-d544-4f71-a758-c8b409f9d9bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:58.066622 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-215e8184-d544-4f71-a758-c8b409f9d9bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:58.067240 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-215e8184-d544-4f71-a758-c8b409f9d9bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:58.068117 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-215e8184-d544-4f71-a758-c8b409f9d9bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:58.068987 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-215e8184-d544-4f71-a758-c8b409f9d9bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:58.069562 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-215e8184-d544-4f71-a758-c8b409f9d9bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:58.274021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-0502d7bf-d309-4370-a0aa-cd30b6cd37e9 req-2cd23d90-d6dd-446f-bef8-e7033c173f48 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "54a16243-ecd7-4152-a1d3-bf92cc3709cc", "name": "network-vif-deleted", "tag": "75592632-690e-4dbf-a132-69141d52e063"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:58.320992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-0502d7bf-d309-4370-a0aa-cd30b6cd37e9 req-2cd23d90-d6dd-446f-bef8-e7033c173f48 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:58.322751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-0502d7bf-d309-4370-a0aa-cd30b6cd37e9 req-2cd23d90-d6dd-446f-bef8-e7033c173f48 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:58.323965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-0502d7bf-d309-4370-a0aa-cd30b6cd37e9 req-2cd23d90-d6dd-446f-bef8-e7033c173f48 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:58.360839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-0502d7bf-d309-4370-a0aa-cd30b6cd37e9 req-2cd23d90-d6dd-446f-bef8-e7033c173f48 service nova] Creating event network-vif-deleted:75592632-690e-4dbf-a132-69141d52e063 for instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc on hv-cinder-maste Aug 07 01:13:58.375871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-0502d7bf-d309-4370-a0aa-cd30b6cd37e9 req-2cd23d90-d6dd-446f-bef8-e7033c173f48 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.110244 Aug 07 01:13:58.377781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 253/488] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:13:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:13:58.402736 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-215e8184-d544-4f71-a758-c8b409f9d9bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:58.409623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:13:58.418713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-93579d95-f513-4ef0-ba9d-45b041607d75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Action: 'create', calling method: .version_select at 0x7f1fad4145e0>, body: {"keypair": {"name": "tempest-keypair-830225765", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEBETfP1UxwmQ+s3lmhoIVQWoD03vZl2r6/Seu2SxUwgdFFWYvFtVAZBzjtRjE3WNxWwWrx0gRvX/jWPm1JBB6LoXeLaO4mleq6gbt9Zbl2LSsXXhIkWEysUazIXXOlhOQ=="}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:13:58.421658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-215e8184-d544-4f71-a758-c8b409f9d9bc tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1678 microversion: 2.1 time: 0.376204 Aug 07 01:13:58.422686 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 236/489] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:58 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1678 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:13:58.456621 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-93579d95-f513-4ef0-ba9d-45b041607d75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Getting quotas for project 9badaa18dcf64810b10dd7bf03d8e40d. Resources: {'key_pairs'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:13:58.464455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-93579d95-f513-4ef0-ba9d-45b041607d75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Getting quotas for user ec931c9cc56a462991413926d34dfa46 and project 9badaa18dcf64810b10dd7bf03d8e40d. Resources: {'key_pairs'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:13:58.546481 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-93579d95-f513-4ef0-ba9d-45b041607d75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.161922 Aug 07 01:13:58.547404 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 254/490] 10.222.0.26 () {60 vars in 1159 bytes} [Sun Aug 7 01:13:58 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:13:59.011297 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-36cd0276-8e76-4043-b1ba-cf4358d958bc tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:59.019630 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-36cd0276-8e76-4043-b1ba-cf4358d958bc tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:59.057812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-36cd0276-8e76-4043-b1ba-cf4358d958bc tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:59.063137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-36cd0276-8e76-4043-b1ba-cf4358d958bc tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:59.065050 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-36cd0276-8e76-4043-b1ba-cf4358d958bc tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:59.067633 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-36cd0276-8e76-4043-b1ba-cf4358d958bc tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:59.073624 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-36cd0276-8e76-4043-b1ba-cf4358d958bc tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:59.074588 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-36cd0276-8e76-4043-b1ba-cf4358d958bc tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:59.129409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-36cd0276-8e76-4043-b1ba-cf4358d958bc tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] HTTP exception thrown: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-36cd0276-8e76-4043-b1ba-cf4358d958bc tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Request method failure captured: Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc HTTP/1.1 Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xD021Vt2D0N359UOKX9xymFYpnZ79FQqLbXwvg5UbTRYuTTlR_HceEoHaOg5fvzy7nSB6FZNa7SVaU3q4lxIRVTXMupRK2mSDBaPiXGNUdpidSOmvxEoLqM4Zox-kqkoV9_K0_oxsudsRHb6jwecdh1rLe0262HFBUWOJmPZoVity8 Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:13:59.132590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:13:59.135052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:13:59.135052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:13:59.135052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:13:59.135052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:13:59.135052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:13:59.135052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:13:59.135052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:13:59.135052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: e65f0f1ba54f483abff788974b1256cb Aug 07 01:13:59.135052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:13:59.135052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:13:59.135052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:13:59.135052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '54a16243-ecd7-4152-a1d3-bf92cc3709cc'} Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:13:59.136468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:13:59.138732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:13:59.143722 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-36cd0276-8e76-4043-b1ba-cf4358d958bc tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Returning 404 to user: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:13:59.145472 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-36cd0276-8e76-4043-b1ba-cf4358d958bc tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 404 len: 111 microversion: 2.1 time: 0.147281 Aug 07 01:13:59.152303 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 237/491] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:58 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 111 bytes in 157 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:13:59.162370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8d011133-51a3-420c-9d2f-3dfb440af37f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '.version_select at 0x7f1fad355430>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:59.219082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8d011133-51a3-420c-9d2f-3dfb440af37f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2078632662" status: 202 len: 0 microversion: 2.1 time: 0.064547 Aug 07 01:13:59.221672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 255/492] 10.222.0.26 () {58 vars in 1221 bytes} [Sun Aug 7 01:13:59 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2078632662 => generated 0 bytes in 68 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:13:59.472644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-acb137be-7cc4-4b7d-9f12-321a5948475f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:13:59.480608 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-acb137be-7cc4-4b7d-9f12-321a5948475f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:13:59.524806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acb137be-7cc4-4b7d-9f12-321a5948475f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:59.527534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acb137be-7cc4-4b7d-9f12-321a5948475f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:59.530051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acb137be-7cc4-4b7d-9f12-321a5948475f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:59.531495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acb137be-7cc4-4b7d-9f12-321a5948475f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:13:59.533165 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acb137be-7cc4-4b7d-9f12-321a5948475f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:13:59.534311 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acb137be-7cc4-4b7d-9f12-321a5948475f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:13:59.839873 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-acb137be-7cc4-4b7d-9f12-321a5948475f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:13:59.856906 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-acb137be-7cc4-4b7d-9f12-321a5948475f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1678 microversion: 2.1 time: 0.392998 Aug 07 01:13:59.860499 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 238/493] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:13:59 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1678 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:00.902169 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-97bac3a9-a32d-45e5-80c5-2eebc164cb6a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:00.904624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-97bac3a9-a32d-45e5-80c5-2eebc164cb6a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:00.923520 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97bac3a9-a32d-45e5-80c5-2eebc164cb6a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:00.925059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97bac3a9-a32d-45e5-80c5-2eebc164cb6a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:00.926320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97bac3a9-a32d-45e5-80c5-2eebc164cb6a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:00.927950 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97bac3a9-a32d-45e5-80c5-2eebc164cb6a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:00.929293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97bac3a9-a32d-45e5-80c5-2eebc164cb6a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:00.930401 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97bac3a9-a32d-45e5-80c5-2eebc164cb6a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:01.020659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3ef0e739-3f87-4144-ad9d-fd0fbf6cfb57 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:01.021975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3ef0e739-3f87-4144-ad9d-fd0fbf6cfb57 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:01.041027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3ef0e739-3f87-4144-ad9d-fd0fbf6cfb57 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:01.042402 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3ef0e739-3f87-4144-ad9d-fd0fbf6cfb57 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:01.043601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3ef0e739-3f87-4144-ad9d-fd0fbf6cfb57 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:01.045115 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3ef0e739-3f87-4144-ad9d-fd0fbf6cfb57 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:01.046095 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3ef0e739-3f87-4144-ad9d-fd0fbf6cfb57 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:01.047072 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3ef0e739-3f87-4144-ad9d-fd0fbf6cfb57 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:01.288508 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.objects.instance [None req-3ef0e739-3f87-4144-ad9d-fd0fbf6cfb57 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lazy-loading 'services' on Instance uuid 96f9a9ce-f29e-48be-a36b-8518126a09ee {{(pid=117984) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:14:01.350614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-97bac3a9-a32d-45e5-80c5-2eebc164cb6a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:01.366833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-97bac3a9-a32d-45e5-80c5-2eebc164cb6a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1678 microversion: 2.1 time: 0.473281 Aug 07 01:14:01.374772 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 256/494] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:00 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1678 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:01.546577 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3ef0e739-3f87-4144-ad9d-fd0fbf6cfb57 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "DELETE /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee/os-volume_attachments/fa48cc70-a60b-440f-8b1a-09ffff661711" status: 202 len: 0 microversion: 2.1 time: 0.535677 Aug 07 01:14:01.553814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 239/495] 10.222.0.26 () {58 vars in 1416 bytes} [Sun Aug 7 01:14:01 2022] DELETE /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee/os-volume_attachments/fa48cc70-a60b-440f-8b1a-09ffff661711 => generated 0 bytes in 544 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:14:01.560186 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0605e01c-0024-4547-bac6-eb3de2fec80a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:01.566906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0605e01c-0024-4547-bac6-eb3de2fec80a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:01.602764 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0605e01c-0024-4547-bac6-eb3de2fec80a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:01.604126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0605e01c-0024-4547-bac6-eb3de2fec80a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:01.604878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0605e01c-0024-4547-bac6-eb3de2fec80a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:01.605986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0605e01c-0024-4547-bac6-eb3de2fec80a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:01.606843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0605e01c-0024-4547-bac6-eb3de2fec80a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:01.607466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0605e01c-0024-4547-bac6-eb3de2fec80a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:01.670757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0605e01c-0024-4547-bac6-eb3de2fec80a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.115371 Aug 07 01:14:01.671833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 257/496] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:14:01 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee/os-volume_attachments => generated 197 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:14:01.861546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-881923885", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "fd0a1aca-5011-496b-8c8b-5bafdabdf653"}], "security_groups": [{"name": "tempest-securitygroup--1514711990"}], "key_name": "tempest-keypair-830225765", "user_data": "}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:02.407180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e16864d1-2a14-49e4-a00e-08a3d042e35e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:02.409000 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e16864d1-2a14-49e4-a00e-08a3d042e35e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:02.422424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e16864d1-2a14-49e4-a00e-08a3d042e35e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:02.423195 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e16864d1-2a14-49e4-a00e-08a3d042e35e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:02.423816 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e16864d1-2a14-49e4-a00e-08a3d042e35e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:02.424812 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e16864d1-2a14-49e4-a00e-08a3d042e35e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:02.425406 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e16864d1-2a14-49e4-a00e-08a3d042e35e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:02.426129 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e16864d1-2a14-49e4-a00e-08a3d042e35e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:02.671146 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e16864d1-2a14-49e4-a00e-08a3d042e35e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:02.690661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e16864d1-2a14-49e4-a00e-08a3d042e35e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1678 microversion: 2.1 time: 0.290528 Aug 07 01:14:02.694509 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 258/497] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:02 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1678 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:02.714427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fbf89fb2-2c78-494d-b75a-415e2afa8158 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:02.718864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fbf89fb2-2c78-494d-b75a-415e2afa8158 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:02.732213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fbf89fb2-2c78-494d-b75a-415e2afa8158 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:02.733548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fbf89fb2-2c78-494d-b75a-415e2afa8158 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:02.734461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fbf89fb2-2c78-494d-b75a-415e2afa8158 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:02.735878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fbf89fb2-2c78-494d-b75a-415e2afa8158 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:02.737040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fbf89fb2-2c78-494d-b75a-415e2afa8158 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:02.738068 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fbf89fb2-2c78-494d-b75a-415e2afa8158 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:02.746580 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:INSERT=2,nova_cell1:SELECT=97,nova_cell1:UPDATE=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:02.806635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fbf89fb2-2c78-494d-b75a-415e2afa8158 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.110017 Aug 07 01:14:02.807410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 259/498] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:14:02 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee/os-volume_attachments => generated 197 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:14:03.720180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ba56ba99-b591-4070-a484-f8b18e8c49f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:03.720922 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ba56ba99-b591-4070-a484-f8b18e8c49f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:03.727753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba56ba99-b591-4070-a484-f8b18e8c49f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:03.728260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba56ba99-b591-4070-a484-f8b18e8c49f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:03.728711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba56ba99-b591-4070-a484-f8b18e8c49f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:03.729435 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba56ba99-b591-4070-a484-f8b18e8c49f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:03.729893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba56ba99-b591-4070-a484-f8b18e8c49f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:03.730344 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba56ba99-b591-4070-a484-f8b18e8c49f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:03.931889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ba56ba99-b591-4070-a484-f8b18e8c49f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:03.947687 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ba56ba99-b591-4070-a484-f8b18e8c49f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1678 microversion: 2.1 time: 0.231379 Aug 07 01:14:03.948556 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 260/499] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:03 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1678 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:03.958686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-66f4f351-a2a9-4693-9236-e997d9c85d6a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:03.963694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-66f4f351-a2a9-4693-9236-e997d9c85d6a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:03.976287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-66f4f351-a2a9-4693-9236-e997d9c85d6a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:03.977064 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-66f4f351-a2a9-4693-9236-e997d9c85d6a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:03.977952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-66f4f351-a2a9-4693-9236-e997d9c85d6a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:03.979266 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-66f4f351-a2a9-4693-9236-e997d9c85d6a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:03.980211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-66f4f351-a2a9-4693-9236-e997d9c85d6a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:03.981024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-66f4f351-a2a9-4693-9236-e997d9c85d6a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:04.020769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-66f4f351-a2a9-4693-9236-e997d9c85d6a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.070142 Aug 07 01:14:04.021360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 261/500] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:14:03 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee/os-volume_attachments => generated 25 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 07 01:14:04.140052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5a397387-1e16-4503-9547-011cf448053a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:04.140926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5a397387-1e16-4503-9547-011cf448053a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:04.149689 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5a397387-1e16-4503-9547-011cf448053a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:04.150254 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5a397387-1e16-4503-9547-011cf448053a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:04.150898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5a397387-1e16-4503-9547-011cf448053a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:04.151664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5a397387-1e16-4503-9547-011cf448053a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:04.152134 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5a397387-1e16-4503-9547-011cf448053a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:04.152606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5a397387-1e16-4503-9547-011cf448053a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:04.184588 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5a397387-1e16-4503-9547-011cf448053a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Going to try to terminate instance {{(pid=117983) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:14:04.295060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5a397387-1e16-4503-9547-011cf448053a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "DELETE /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 204 len: 0 microversion: 2.1 time: 0.159754 Aug 07 01:14:04.295974 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 262/501] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:14:04 2022] DELETE /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 0 bytes in 162 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 07 01:14:04.306512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-05ed17d4-e57f-4b1d-a120-c9e4fb816fa0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:04.307348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-05ed17d4-e57f-4b1d-a120-c9e4fb816fa0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:04.316092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05ed17d4-e57f-4b1d-a120-c9e4fb816fa0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:04.316621 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05ed17d4-e57f-4b1d-a120-c9e4fb816fa0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:04.317168 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05ed17d4-e57f-4b1d-a120-c9e4fb816fa0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:04.318039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05ed17d4-e57f-4b1d-a120-c9e4fb816fa0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:04.318502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05ed17d4-e57f-4b1d-a120-c9e4fb816fa0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:04.318885 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05ed17d4-e57f-4b1d-a120-c9e4fb816fa0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:04.526323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-05ed17d4-e57f-4b1d-a120-c9e4fb816fa0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:04.537300 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-05ed17d4-e57f-4b1d-a120-c9e4fb816fa0 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1682 microversion: 2.1 time: 0.235341 Aug 07 01:14:04.538167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 263/502] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:04 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1682 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:04.978177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-19531b54-3ab4-482c-91fd-3404af6c7622 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:04.979398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-19531b54-3ab4-482c-91fd-3404af6c7622 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:04.993442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19531b54-3ab4-482c-91fd-3404af6c7622 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:04.994371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19531b54-3ab4-482c-91fd-3404af6c7622 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:04.995211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19531b54-3ab4-482c-91fd-3404af6c7622 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:04.996611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19531b54-3ab4-482c-91fd-3404af6c7622 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:04.997592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19531b54-3ab4-482c-91fd-3404af6c7622 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:04.998566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19531b54-3ab4-482c-91fd-3404af6c7622 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:05.231850 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-19531b54-3ab4-482c-91fd-3404af6c7622 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:05.241293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-19531b54-3ab4-482c-91fd-3404af6c7622 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1678 microversion: 2.1 time: 0.269402 Aug 07 01:14:05.241293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 264/503] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:04 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1678 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:05.562437 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e03fda43-57f1-426c-a45c-0258b2feeb9a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:05.563094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e03fda43-57f1-426c-a45c-0258b2feeb9a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:05.571161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e03fda43-57f1-426c-a45c-0258b2feeb9a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:05.571920 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e03fda43-57f1-426c-a45c-0258b2feeb9a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:05.572450 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e03fda43-57f1-426c-a45c-0258b2feeb9a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:05.573190 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e03fda43-57f1-426c-a45c-0258b2feeb9a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:05.573690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e03fda43-57f1-426c-a45c-0258b2feeb9a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:05.574127 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e03fda43-57f1-426c-a45c-0258b2feeb9a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:05.792339 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e03fda43-57f1-426c-a45c-0258b2feeb9a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:05.802933 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e03fda43-57f1-426c-a45c-0258b2feeb9a tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1682 microversion: 2.1 time: 0.244592 Aug 07 01:14:05.803531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 265/504] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:05 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1682 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:06.017494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.network.neutron [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] validate_networks() for [('fd0a1aca-5011-496b-8c8b-5bafdabdf653', None, None, None, None, None)] {{(pid=117984) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:14:06.276704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-03e2b7de-4c2e-4502-84e7-5f0233943811 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:06.278963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-03e2b7de-4c2e-4502-84e7-5f0233943811 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:06.295465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-03e2b7de-4c2e-4502-84e7-5f0233943811 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:06.296436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-03e2b7de-4c2e-4502-84e7-5f0233943811 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:06.297178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-03e2b7de-4c2e-4502-84e7-5f0233943811 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:06.299575 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-03e2b7de-4c2e-4502-84e7-5f0233943811 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:06.300772 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-03e2b7de-4c2e-4502-84e7-5f0233943811 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:06.302055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-03e2b7de-4c2e-4502-84e7-5f0233943811 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:06.634899 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-03e2b7de-4c2e-4502-84e7-5f0233943811 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:06.650897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-03e2b7de-4c2e-4502-84e7-5f0233943811 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1655 microversion: 2.1 time: 0.388502 Aug 07 01:14:06.651810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 266/505] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:06 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1655 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:06.698329 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8232d81e-e8ea-4ca4-bd52-fe1e73f8ed2f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:06.709572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8232d81e-e8ea-4ca4-bd52-fe1e73f8ed2f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:06.710434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8232d81e-e8ea-4ca4-bd52-fe1e73f8ed2f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:06.711197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8232d81e-e8ea-4ca4-bd52-fe1e73f8ed2f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:06.762121 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8232d81e-e8ea-4ca4-bd52-fe1e73f8ed2f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Going to try to start instance {{(pid=117983) start /opt/stack/nova/nova/compute/api.py:2732}} Aug 07 01:14:06.773551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Flavor limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:14:06.774345 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Image limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:14:06.775519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Flavor pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:14:06.776748 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Image pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:14:06.777564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:14:06.801554 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:06.802743 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:06.803737 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:06.805424 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:06.806933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:06.808558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:06.850143 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Getting quotas for project 9badaa18dcf64810b10dd7bf03d8e40d. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:14:06.851934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8232d81e-e8ea-4ca4-bd52-fe1e73f8ed2f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "POST /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/action" status: 202 len: 0 microversion: 2.1 time: 0.160497 Aug 07 01:14:06.853139 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 267/506] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:14:06 2022] POST /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/action => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:14:06.864370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-aa9d3032-9334-48e3-acc1-52b446cd2d5e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:06.866422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-aa9d3032-9334-48e3-acc1-52b446cd2d5e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:06.873617 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Getting quotas for user ec931c9cc56a462991413926d34dfa46 and project 9badaa18dcf64810b10dd7bf03d8e40d. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:14:06.882145 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa9d3032-9334-48e3-acc1-52b446cd2d5e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:06.883066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa9d3032-9334-48e3-acc1-52b446cd2d5e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:06.885066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa9d3032-9334-48e3-acc1-52b446cd2d5e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:06.887323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa9d3032-9334-48e3-acc1-52b446cd2d5e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:06.888529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa9d3032-9334-48e3-acc1-52b446cd2d5e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:06.889700 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa9d3032-9334-48e3-acc1-52b446cd2d5e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:06.908879 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Going to run 1 instances... {{(pid=117984) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:14:06.913485 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117984) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:14:07.003083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a0ce64ea-3368-4819-85f1-ee70467a8283 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 5.150513 Aug 07 01:14:07.004221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 240/507] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:14:01 2022] POST /compute/v2.1/servers => generated 409 bytes in 5153 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:14:07.009971 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e4896bc9-1d0a-4bc5-9190-62a9e916d64c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:07.010705 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e4896bc9-1d0a-4bc5-9190-62a9e916d64c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:07.019089 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4896bc9-1d0a-4bc5-9190-62a9e916d64c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:07.020928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4896bc9-1d0a-4bc5-9190-62a9e916d64c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:07.021751 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4896bc9-1d0a-4bc5-9190-62a9e916d64c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:07.022807 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4896bc9-1d0a-4bc5-9190-62a9e916d64c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:07.023490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4896bc9-1d0a-4bc5-9190-62a9e916d64c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:07.024716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4896bc9-1d0a-4bc5-9190-62a9e916d64c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:07.427221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e4896bc9-1d0a-4bc5-9190-62a9e916d64c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:07.452490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e4896bc9-1d0a-4bc5-9190-62a9e916d64c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1664 microversion: 2.1 time: 0.444793 Aug 07 01:14:07.456333 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 241/508] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:07 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1664 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:07.467690 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-39cbb0b4-0d56-4881-abcb-cf55c7ef5fe3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:07.470241 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-39cbb0b4-0d56-4881-abcb-cf55c7ef5fe3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:07.572960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-aa9d3032-9334-48e3-acc1-52b446cd2d5e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:07.584230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-aa9d3032-9334-48e3-acc1-52b446cd2d5e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1682 microversion: 2.1 time: 0.728972 Aug 07 01:14:07.585652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 268/509] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:06 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1682 bytes in 731 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:07.597673 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-39cbb0b4-0d56-4881-abcb-cf55c7ef5fe3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:07.611822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-39cbb0b4-0d56-4881-abcb-cf55c7ef5fe3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1244 microversion: 2.1 time: 0.152711 Aug 07 01:14:07.613505 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 242/510] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:07 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1244 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:08.370945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-30fec164-2b0a-4fd3-a927-968418096e31 req-e01a96ea-73f2-4e05-ac8c-d2586f4c5eeb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "96f9a9ce-f29e-48be-a36b-8518126a09ee", "name": "network-vif-deleted", "tag": "f71ffe87-943a-40bf-94f2-b17532b7092c"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:08.394738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-30fec164-2b0a-4fd3-a927-968418096e31 req-e01a96ea-73f2-4e05-ac8c-d2586f4c5eeb service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:08.395342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-30fec164-2b0a-4fd3-a927-968418096e31 req-e01a96ea-73f2-4e05-ac8c-d2586f4c5eeb service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:08.395679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-30fec164-2b0a-4fd3-a927-968418096e31 req-e01a96ea-73f2-4e05-ac8c-d2586f4c5eeb service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:08.420480 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-30fec164-2b0a-4fd3-a927-968418096e31 req-e01a96ea-73f2-4e05-ac8c-d2586f4c5eeb service nova] Creating event network-vif-deleted:f71ffe87-943a-40bf-94f2-b17532b7092c for instance 96f9a9ce-f29e-48be-a36b-8518126a09ee on hv-cinder-maste Aug 07 01:14:08.434194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-30fec164-2b0a-4fd3-a927-968418096e31 req-e01a96ea-73f2-4e05-ac8c-d2586f4c5eeb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069717 Aug 07 01:14:08.435531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 269/511] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:14:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:14:08.479338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d8c9c413-7cce-4499-ba40-f1a026a6e98c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:08.480202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d8c9c413-7cce-4499-ba40-f1a026a6e98c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:08.490202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8c9c413-7cce-4499-ba40-f1a026a6e98c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:08.491265 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8c9c413-7cce-4499-ba40-f1a026a6e98c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:08.492034 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8c9c413-7cce-4499-ba40-f1a026a6e98c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:08.493824 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8c9c413-7cce-4499-ba40-f1a026a6e98c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:08.494459 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8c9c413-7cce-4499-ba40-f1a026a6e98c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:08.495044 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8c9c413-7cce-4499-ba40-f1a026a6e98c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:08.623969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8eee303a-b138-4da8-87c7-0900cd87f2d7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:08.624836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8eee303a-b138-4da8-87c7-0900cd87f2d7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:08.626791 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=4 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:08.634048 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8eee303a-b138-4da8-87c7-0900cd87f2d7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:08.634592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8eee303a-b138-4da8-87c7-0900cd87f2d7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:08.635032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8eee303a-b138-4da8-87c7-0900cd87f2d7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:08.635830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8eee303a-b138-4da8-87c7-0900cd87f2d7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:08.636314 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8eee303a-b138-4da8-87c7-0900cd87f2d7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:08.636735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8eee303a-b138-4da8-87c7-0900cd87f2d7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:08.694168 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8eee303a-b138-4da8-87c7-0900cd87f2d7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:08.695429 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8eee303a-b138-4da8-87c7-0900cd87f2d7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:08.696345 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8eee303a-b138-4da8-87c7-0900cd87f2d7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:08.697514 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.objects.instance [None req-8eee303a-b138-4da8-87c7-0900cd87f2d7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lazy-loading 'fault' on Instance uuid 96f9a9ce-f29e-48be-a36b-8518126a09ee {{(pid=117983) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:14:08.698505 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d8c9c413-7cce-4499-ba40-f1a026a6e98c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:08.715812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d8c9c413-7cce-4499-ba40-f1a026a6e98c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1664 microversion: 2.1 time: 0.240605 Aug 07 01:14:08.723152 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 243/512] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:08 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1664 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:08.723152 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f85e695c-0107-4058-ae9f-ce683a84ae80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:08.723152 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f85e695c-0107-4058-ae9f-ce683a84ae80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:08.729401 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f85e695c-0107-4058-ae9f-ce683a84ae80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:08.730206 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f85e695c-0107-4058-ae9f-ce683a84ae80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:08.730835 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f85e695c-0107-4058-ae9f-ce683a84ae80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:08.731723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f85e695c-0107-4058-ae9f-ce683a84ae80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:08.732389 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f85e695c-0107-4058-ae9f-ce683a84ae80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:08.732999 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f85e695c-0107-4058-ae9f-ce683a84ae80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:08.785863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8eee303a-b138-4da8-87c7-0900cd87f2d7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:08.795100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8eee303a-b138-4da8-87c7-0900cd87f2d7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1331 microversion: 2.1 time: 0.175506 Aug 07 01:14:08.795864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 270/513] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:08 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1331 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:08.820944 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f85e695c-0107-4058-ae9f-ce683a84ae80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:08.834861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f85e695c-0107-4058-ae9f-ce683a84ae80 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1296 microversion: 2.1 time: 0.116744 Aug 07 01:14:08.836447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 244/514] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:08 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1296 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:09.741396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fb7df4fd-3620-47ad-97f7-89bd11a399b9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:09.742072 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fb7df4fd-3620-47ad-97f7-89bd11a399b9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:09.748957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fb7df4fd-3620-47ad-97f7-89bd11a399b9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:09.749477 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fb7df4fd-3620-47ad-97f7-89bd11a399b9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:09.749925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fb7df4fd-3620-47ad-97f7-89bd11a399b9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:09.751040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fb7df4fd-3620-47ad-97f7-89bd11a399b9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:09.751523 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fb7df4fd-3620-47ad-97f7-89bd11a399b9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:09.751966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fb7df4fd-3620-47ad-97f7-89bd11a399b9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:09.822107 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ec733368-5f93-450b-a387-5394f484a163 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:09.823193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ec733368-5f93-450b-a387-5394f484a163 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:09.835033 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec733368-5f93-450b-a387-5394f484a163 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:09.835629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec733368-5f93-450b-a387-5394f484a163 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:09.836579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec733368-5f93-450b-a387-5394f484a163 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:09.837914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec733368-5f93-450b-a387-5394f484a163 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:09.838738 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec733368-5f93-450b-a387-5394f484a163 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:09.839407 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec733368-5f93-450b-a387-5394f484a163 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:09.897350 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec733368-5f93-450b-a387-5394f484a163 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:09.898009 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec733368-5f93-450b-a387-5394f484a163 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:09.898592 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec733368-5f93-450b-a387-5394f484a163 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:09.899186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.objects.instance [None req-ec733368-5f93-450b-a387-5394f484a163 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lazy-loading 'fault' on Instance uuid 96f9a9ce-f29e-48be-a36b-8518126a09ee {{(pid=117984) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:14:09.980137 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-fb7df4fd-3620-47ad-97f7-89bd11a399b9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:10.006604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fb7df4fd-3620-47ad-97f7-89bd11a399b9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1668 microversion: 2.1 time: 0.269616 Aug 07 01:14:10.007617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 271/515] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:09 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1668 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:10.008438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ec733368-5f93-450b-a387-5394f484a163 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:10.029987 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ec733368-5f93-450b-a387-5394f484a163 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 200 len: 1331 microversion: 2.1 time: 0.214639 Aug 07 01:14:10.031153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-55d88845-2d5f-4ef7-8e4a-3cfcd7a17474 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:10.032180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-55d88845-2d5f-4ef7-8e4a-3cfcd7a17474 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:10.036899 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 245/516] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:09 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 1331 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:10.052901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-55d88845-2d5f-4ef7-8e4a-3cfcd7a17474 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:10.053876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-55d88845-2d5f-4ef7-8e4a-3cfcd7a17474 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:10.054808 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-55d88845-2d5f-4ef7-8e4a-3cfcd7a17474 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:10.056168 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-55d88845-2d5f-4ef7-8e4a-3cfcd7a17474 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:10.057268 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-55d88845-2d5f-4ef7-8e4a-3cfcd7a17474 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:10.057852 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-55d88845-2d5f-4ef7-8e4a-3cfcd7a17474 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:10.237479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-55d88845-2d5f-4ef7-8e4a-3cfcd7a17474 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:10.254515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-55d88845-2d5f-4ef7-8e4a-3cfcd7a17474 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1304 microversion: 2.1 time: 0.235871 Aug 07 01:14:10.255410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 272/517] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:10 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1304 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:11.056412 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e10a7ac2-158a-4b1a-b589-e6c3614ce494 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:11.057745 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e10a7ac2-158a-4b1a-b589-e6c3614ce494 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:11.069422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e10a7ac2-158a-4b1a-b589-e6c3614ce494 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:11.070811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e10a7ac2-158a-4b1a-b589-e6c3614ce494 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:11.070811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e10a7ac2-158a-4b1a-b589-e6c3614ce494 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:11.071956 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e10a7ac2-158a-4b1a-b589-e6c3614ce494 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:11.072847 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e10a7ac2-158a-4b1a-b589-e6c3614ce494 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:11.073476 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e10a7ac2-158a-4b1a-b589-e6c3614ce494 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:11.092315 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-e10a7ac2-158a-4b1a-b589-e6c3614ce494 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] HTTP exception thrown: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e10a7ac2-158a-4b1a-b589-e6c3614ce494 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Request method failure captured: Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee HTTP/1.1 Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xD021Vt2D0N359UOKX9xymFYpnZ79FQqLbXwvg5UbTRYuTTlR_HceEoHaOg5fvzy7nSB6FZNa7SVaU3q4lxIRVTXMupRK2mSDBaPiXGNUdpidSOmvxEoLqM4Zox-kqkoV9_K0_oxsudsRHb6jwecdh1rLe0262HFBUWOJmPZoVity8 Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:14:11.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:14:11.097042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:14:11.097042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:14:11.097042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:14:11.097042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:14:11.097042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:14:11.097042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:14:11.097042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:14:11.097042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: e65f0f1ba54f483abff788974b1256cb Aug 07 01:14:11.097042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:14:11.097042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:14:11.097042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:14:11.097042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '96f9a9ce-f29e-48be-a36b-8518126a09ee'} Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:14:11.098048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:14:11.099446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:14:11.100891 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e10a7ac2-158a-4b1a-b589-e6c3614ce494 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Returning 404 to user: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:14:11.101319 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e10a7ac2-158a-4b1a-b589-e6c3614ce494 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 404 len: 111 microversion: 2.1 time: 0.050699 Aug 07 01:14:11.106253 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 246/518] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:11 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:14:11.112759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-16066339-935d-4629-9349-c0e0409a60e9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '.version_select at 0x7f1fad6eb430>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:11.136851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-16066339-935d-4629-9349-c0e0409a60e9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1615386904" status: 202 len: 0 microversion: 2.1 time: 0.028658 Aug 07 01:14:11.138659 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 273/519] 10.222.0.26 () {58 vars in 1221 bytes} [Sun Aug 7 01:14:11 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1615386904 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:14:11.283831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6885fa23-9e2f-4311-899a-4539a2048de8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:11.286081 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6885fa23-9e2f-4311-899a-4539a2048de8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:11.309161 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6885fa23-9e2f-4311-899a-4539a2048de8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:11.310422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6885fa23-9e2f-4311-899a-4539a2048de8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:11.311226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6885fa23-9e2f-4311-899a-4539a2048de8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:11.312599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6885fa23-9e2f-4311-899a-4539a2048de8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:11.313528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6885fa23-9e2f-4311-899a-4539a2048de8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:11.314358 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6885fa23-9e2f-4311-899a-4539a2048de8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:11.596996 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6885fa23-9e2f-4311-899a-4539a2048de8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:11.611404 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6885fa23-9e2f-4311-899a-4539a2048de8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1370 microversion: 2.1 time: 0.332110 Aug 07 01:14:11.612564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 247/520] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:11 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1370 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:12.481032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f30490b9-f0e8-4e77-a1e9-4d995c4c5f1c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Action: 'create', calling method: .version_select at 0x7f1fad21aca0>, body: {"keypair": {"name": "tempest-keypair-1298711506", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNTFNWgTdO1b9RbLSbbEew4huQrL98SDroQ81a7i8eVU+fvToCWIgHjR8o6krYwY9ksYBK0+s14ipcL7hTuyv7tFkRqCubC7TSRUKu6GOQSk6LmQ00TINxyrzKv644xACg=="}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:12.505062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-f30490b9-f0e8-4e77-a1e9-4d995c4c5f1c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for project 35820b4f79774ac49f43fce915f56d6f. Resources: {'key_pairs'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:14:12.511479 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-00d73b1e-b9b4-4bb5-a80f-cab9fbe0a0f9 req-b70f1993-ea46-4222-8b3a-2c18e22d47ee service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6ac28640-d6da-4cfc-b7ad-0d69136ac1fd", "tag": "4d7d0711-c6ef-4ae2-b272-c2cf175993d4"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:12.522467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=86,nova_api:INSERT=4 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:12.529106 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-00d73b1e-b9b4-4bb5-a80f-cab9fbe0a0f9 req-b70f1993-ea46-4222-8b3a-2c18e22d47ee service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:12.529848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-00d73b1e-b9b4-4bb5-a80f-cab9fbe0a0f9 req-b70f1993-ea46-4222-8b3a-2c18e22d47ee service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:12.530411 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-f30490b9-f0e8-4e77-a1e9-4d995c4c5f1c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for user e65f0f1ba54f483abff788974b1256cb and project 35820b4f79774ac49f43fce915f56d6f. Resources: {'key_pairs'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:14:12.530720 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-00d73b1e-b9b4-4bb5-a80f-cab9fbe0a0f9 req-b70f1993-ea46-4222-8b3a-2c18e22d47ee service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:12.544030 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-00d73b1e-b9b4-4bb5-a80f-cab9fbe0a0f9 req-b70f1993-ea46-4222-8b3a-2c18e22d47ee service nova] Creating event network-changed:4d7d0711-c6ef-4ae2-b272-c2cf175993d4 for instance 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd on hv-cinder-maste Aug 07 01:14:12.550520 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=2,nova_api:INSERT=1,nova_api:DELETE=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:12.551814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-00d73b1e-b9b4-4bb5-a80f-cab9fbe0a0f9 req-b70f1993-ea46-4222-8b3a-2c18e22d47ee service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044189 Aug 07 01:14:12.552860 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 248/521] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:14:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:14:12.575749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f30490b9-f0e8-4e77-a1e9-4d995c4c5f1c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.099273 Aug 07 01:14:12.577323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 274/522] 10.222.0.26 () {60 vars in 1159 bytes} [Sun Aug 7 01:14:12 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:14:12.636196 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6a94bbe2-5b46-4382-be4f-accbbf506916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:12.637379 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6a94bbe2-5b46-4382-be4f-accbbf506916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:12.647101 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6a94bbe2-5b46-4382-be4f-accbbf506916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:12.652875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6a94bbe2-5b46-4382-be4f-accbbf506916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:12.653866 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6a94bbe2-5b46-4382-be4f-accbbf506916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:12.655487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6a94bbe2-5b46-4382-be4f-accbbf506916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:12.656550 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6a94bbe2-5b46-4382-be4f-accbbf506916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:12.657969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6a94bbe2-5b46-4382-be4f-accbbf506916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:13.215171 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6a94bbe2-5b46-4382-be4f-accbbf506916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:13.225942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6a94bbe2-5b46-4382-be4f-accbbf506916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1370 microversion: 2.1 time: 0.594502 Aug 07 01:14:13.226662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 249/523] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:12 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1370 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:14.249667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0074e3c8-7b7a-43b9-85c6-ef414e81b974 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:14.250456 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0074e3c8-7b7a-43b9-85c6-ef414e81b974 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:14.260147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0074e3c8-7b7a-43b9-85c6-ef414e81b974 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:14.261452 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0074e3c8-7b7a-43b9-85c6-ef414e81b974 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:14.262252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0074e3c8-7b7a-43b9-85c6-ef414e81b974 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:14.263553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0074e3c8-7b7a-43b9-85c6-ef414e81b974 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:14.264421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0074e3c8-7b7a-43b9-85c6-ef414e81b974 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:14.265166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0074e3c8-7b7a-43b9-85c6-ef414e81b974 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:14.295911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:14.659125 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0074e3c8-7b7a-43b9-85c6-ef414e81b974 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:14.669367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0074e3c8-7b7a-43b9-85c6-ef414e81b974 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1370 microversion: 2.1 time: 0.425352 Aug 07 01:14:14.670072 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 275/524] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:14 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1370 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:14.801924 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-590b106c-3e35-4df8-b38d-ddc6996f42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5ed02bd8-a8d9-42b9-9d85-f4c9fe9f60ff", "device": "/dev/sdb"}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:14.807408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-590b106c-3e35-4df8-b38d-ddc6996f42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:14.820503 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-590b106c-3e35-4df8-b38d-ddc6996f42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:14.821467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-590b106c-3e35-4df8-b38d-ddc6996f42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:14.822433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-590b106c-3e35-4df8-b38d-ddc6996f42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:14.823727 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-590b106c-3e35-4df8-b38d-ddc6996f42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:14.824585 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-590b106c-3e35-4df8-b38d-ddc6996f42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:14.825360 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-590b106c-3e35-4df8-b38d-ddc6996f42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:15.235402 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-590b106c-3e35-4df8-b38d-ddc6996f42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "POST /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.440621 Aug 07 01:14:15.236763 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 250/525] 10.222.0.26 () {60 vars in 1323 bytes} [Sun Aug 7 01:14:14 2022] POST /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc/os-volume_attachments => generated 194 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:14:15.713989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-56e9ab7f-4f95-4133-9c16-e6887147bd9e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:15.714976 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-56e9ab7f-4f95-4133-9c16-e6887147bd9e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:15.717277 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1930209970", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "04909372-4f17-4c44-b384-6e7c31434dac"}], "security_groups": [{"name": "tempest-securitygroup--488552700"}], "key_name": "tempest-keypair-1298711506", "user_data": "}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:15.727446 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-56e9ab7f-4f95-4133-9c16-e6887147bd9e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:15.728209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-56e9ab7f-4f95-4133-9c16-e6887147bd9e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:15.728945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-56e9ab7f-4f95-4133-9c16-e6887147bd9e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:15.730219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-56e9ab7f-4f95-4133-9c16-e6887147bd9e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:15.731046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-56e9ab7f-4f95-4133-9c16-e6887147bd9e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:15.732092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-56e9ab7f-4f95-4133-9c16-e6887147bd9e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:15.879704 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.network.neutron [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] validate_networks() for [('04909372-4f17-4c44-b384-6e7c31434dac', None, None, None, None, None)] {{(pid=117984) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:14:15.964017 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-56e9ab7f-4f95-4133-9c16-e6887147bd9e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:15.976143 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-56e9ab7f-4f95-4133-9c16-e6887147bd9e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1370 microversion: 2.1 time: 0.274115 Aug 07 01:14:15.977487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 276/526] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:15 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1370 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:16.261252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-519215402", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1022328377", "security_groups": [{"name": "tempest-secgroup-smoke-1560529481"}], "block_device_mapping_v2": [{"uuid": "a3609554-3880-4709-ac8d-b5fc7bfeae66", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "14132cfd-b7a7-4ed9-b2b4-817954fdbf3b"}]}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:16.457334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Flavor limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:14:16.459356 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Image limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:14:16.460617 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Flavor pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:14:16.461593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Image pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:14:16.462616 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:14:16.483074 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:16.485453 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:16.487284 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:16.489159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:16.490308 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:16.491052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:16.527040 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for project 35820b4f79774ac49f43fce915f56d6f. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:14:16.536706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for user e65f0f1ba54f483abff788974b1256cb and project 35820b4f79774ac49f43fce915f56d6f. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:14:16.561962 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Going to run 1 instances... {{(pid=117984) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:14:16.567159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117984) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:14:16.664520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b1618d10-451e-4fb2-af72-7c2dfd002254 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.955032 Aug 07 01:14:16.671589 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 251/527] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:14:15 2022] POST /compute/v2.1/servers => generated 401 bytes in 958 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:14:16.682184 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5f49d657-4d0a-490e-b577-d4371ef45eb9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:16.683305 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5f49d657-4d0a-490e-b577-d4371ef45eb9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:16.697355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.network.neutron [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] validate_networks() for [('14132cfd-b7a7-4ed9-b2b4-817954fdbf3b', None, None, None, None, None)] {{(pid=117983) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:14:16.769468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5f49d657-4d0a-490e-b577-d4371ef45eb9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:16.781874 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5f49d657-4d0a-490e-b577-d4371ef45eb9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1242 microversion: 2.1 time: 0.107291 Aug 07 01:14:16.782973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 252/528] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:16 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1242 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:17.002576 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b947360f-5942-43b0-b66b-252855c2064f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:17.004060 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b947360f-5942-43b0-b66b-252855c2064f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:17.013865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b947360f-5942-43b0-b66b-252855c2064f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:17.014763 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b947360f-5942-43b0-b66b-252855c2064f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:17.015812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b947360f-5942-43b0-b66b-252855c2064f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:17.018094 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b947360f-5942-43b0-b66b-252855c2064f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:17.018864 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b947360f-5942-43b0-b66b-252855c2064f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:17.019607 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b947360f-5942-43b0-b66b-252855c2064f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:17.286416 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b947360f-5942-43b0-b66b-252855c2064f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:17.299020 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b947360f-5942-43b0-b66b-252855c2064f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1370 microversion: 2.1 time: 0.301343 Aug 07 01:14:17.299759 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 253/529] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:16 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1370 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:17.355766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Flavor limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:14:17.356917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Image limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:14:17.358122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Flavor pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:14:17.358953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Image pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:14:17.359511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:14:17.375279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:17.376996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:17.378393 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:17.380011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:17.380945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:17.382317 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:17.432693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Getting quotas for project b73b98c606a34d1faf0a31a9f6c59e56. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:14:17.447371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Getting quotas for user 642126380bcf4446bf1e320f93139034 and project b73b98c606a34d1faf0a31a9f6c59e56. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:14:17.471004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-2da7ef95-7558-4787-ae43-c835a25af76d req-b5aeb6f6-ec37-4855-88c3-0cb6b0267490 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6ac28640-d6da-4cfc-b7ad-0d69136ac1fd", "name": "network-vif-plugged", "status": "completed", "tag": "4d7d0711-c6ef-4ae2-b272-c2cf175993d4"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:17.489828 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-2da7ef95-7558-4787-ae43-c835a25af76d req-b5aeb6f6-ec37-4855-88c3-0cb6b0267490 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:17.490825 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-2da7ef95-7558-4787-ae43-c835a25af76d req-b5aeb6f6-ec37-4855-88c3-0cb6b0267490 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:17.492117 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-2da7ef95-7558-4787-ae43-c835a25af76d req-b5aeb6f6-ec37-4855-88c3-0cb6b0267490 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:17.516070 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-2da7ef95-7558-4787-ae43-c835a25af76d req-b5aeb6f6-ec37-4855-88c3-0cb6b0267490 service nova] Creating event network-vif-plugged:4d7d0711-c6ef-4ae2-b272-c2cf175993d4 for instance 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd on hv-cinder-maste Aug 07 01:14:17.527289 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-2da7ef95-7558-4787-ae43-c835a25af76d req-b5aeb6f6-ec37-4855-88c3-0cb6b0267490 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062505 Aug 07 01:14:17.529875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 254/530] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:14:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:14:17.556346 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Going to run 1 instances... {{(pid=117983) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:14:17.559757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='a3609554-3880-4709-ac8d-b5fc7bfeae66',volume_size=None,volume_type=None)] {{(pid=117983) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:14:17.823428 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-65076354-dec2-4957-a525-e426143c3052 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:17.824455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e0975ae3-4c31-487b-8ba1-dadf10088273 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.556184 Aug 07 01:14:17.831699 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-65076354-dec2-4957-a525-e426143c3052 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:17.838592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 277/531] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:14:16 2022] POST /compute/v2.1/servers => generated 406 bytes in 1586 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:14:17.847869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7222c49a-b8b8-4e82-a2e1-ec06aeabdb05 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:17.849832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7222c49a-b8b8-4e82-a2e1-ec06aeabdb05 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:17.850729 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65076354-dec2-4957-a525-e426143c3052 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:17.851949 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65076354-dec2-4957-a525-e426143c3052 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:17.852810 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65076354-dec2-4957-a525-e426143c3052 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:17.854269 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65076354-dec2-4957-a525-e426143c3052 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:17.855617 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65076354-dec2-4957-a525-e426143c3052 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:17.856346 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65076354-dec2-4957-a525-e426143c3052 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:17.991951 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7222c49a-b8b8-4e82-a2e1-ec06aeabdb05 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:18.006632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7222c49a-b8b8-4e82-a2e1-ec06aeabdb05 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1094 microversion: 2.1 time: 0.165411 Aug 07 01:14:18.013077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 278/532] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:17 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1094 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:18.014555 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-65076354-dec2-4957-a525-e426143c3052 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:18.029149 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-65076354-dec2-4957-a525-e426143c3052 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1294 microversion: 2.1 time: 0.213601 Aug 07 01:14:18.030255 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 255/533] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:17 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1294 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:18.336578 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0931e2db-60ff-4495-9384-36a8e112bc42 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:18.337428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0931e2db-60ff-4495-9384-36a8e112bc42 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:18.346355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0931e2db-60ff-4495-9384-36a8e112bc42 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:18.347541 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0931e2db-60ff-4495-9384-36a8e112bc42 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:18.348322 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0931e2db-60ff-4495-9384-36a8e112bc42 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:18.349940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0931e2db-60ff-4495-9384-36a8e112bc42 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:18.351230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0931e2db-60ff-4495-9384-36a8e112bc42 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:18.352466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0931e2db-60ff-4495-9384-36a8e112bc42 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:18.589678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0931e2db-60ff-4495-9384-36a8e112bc42 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:18.601089 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0931e2db-60ff-4495-9384-36a8e112bc42 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1370 microversion: 2.1 time: 0.269449 Aug 07 01:14:18.601679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 279/534] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:18 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1370 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:19.044542 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b2408412-defe-4c08-b205-da5a65c93e8c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:19.045723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b2408412-defe-4c08-b205-da5a65c93e8c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:19.058122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b2408412-defe-4c08-b205-da5a65c93e8c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:19.058985 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b2408412-defe-4c08-b205-da5a65c93e8c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:19.059794 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b2408412-defe-4c08-b205-da5a65c93e8c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:19.061162 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b2408412-defe-4c08-b205-da5a65c93e8c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:19.061973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b2408412-defe-4c08-b205-da5a65c93e8c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:19.062734 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b2408412-defe-4c08-b205-da5a65c93e8c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:19.069808 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d0e2f0ac-433e-4200-a4be-48af24126188 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:19.070580 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d0e2f0ac-433e-4200-a4be-48af24126188 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:19.084077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0e2f0ac-433e-4200-a4be-48af24126188 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:19.085104 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0e2f0ac-433e-4200-a4be-48af24126188 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:19.085920 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0e2f0ac-433e-4200-a4be-48af24126188 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:19.087220 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0e2f0ac-433e-4200-a4be-48af24126188 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:19.088066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0e2f0ac-433e-4200-a4be-48af24126188 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:19.088947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0e2f0ac-433e-4200-a4be-48af24126188 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:19.185401 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b2408412-defe-4c08-b205-da5a65c93e8c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:19.208002 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b2408412-defe-4c08-b205-da5a65c93e8c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1136 microversion: 2.1 time: 0.171154 Aug 07 01:14:19.209406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 256/535] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:19 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1136 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:19.237895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d0e2f0ac-433e-4200-a4be-48af24126188 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:19.254591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d0e2f0ac-433e-4200-a4be-48af24126188 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1312 microversion: 2.1 time: 0.192591 Aug 07 01:14:19.256264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 280/536] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:19 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1312 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:19.650766 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bc9786ea-a28d-4b94-bab2-d797d6800108 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:19.654931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bc9786ea-a28d-4b94-bab2-d797d6800108 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:19.674522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc9786ea-a28d-4b94-bab2-d797d6800108 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:19.677402 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc9786ea-a28d-4b94-bab2-d797d6800108 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:19.678438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc9786ea-a28d-4b94-bab2-d797d6800108 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:19.680689 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc9786ea-a28d-4b94-bab2-d797d6800108 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:19.682189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc9786ea-a28d-4b94-bab2-d797d6800108 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:19.683347 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc9786ea-a28d-4b94-bab2-d797d6800108 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:19.697706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:19.945801 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bc9786ea-a28d-4b94-bab2-d797d6800108 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:19.955316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bc9786ea-a28d-4b94-bab2-d797d6800108 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1370 microversion: 2.1 time: 0.315947 Aug 07 01:14:19.955941 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 257/537] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:19 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1370 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:20.250490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7af73e37-4673-43af-9963-9c3c393fd037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:20.253022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7af73e37-4673-43af-9963-9c3c393fd037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:20.275659 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7af73e37-4673-43af-9963-9c3c393fd037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:20.276264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7af73e37-4673-43af-9963-9c3c393fd037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:20.277005 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7af73e37-4673-43af-9963-9c3c393fd037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:20.279188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7af73e37-4673-43af-9963-9c3c393fd037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:20.280361 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7af73e37-4673-43af-9963-9c3c393fd037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:20.281407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7af73e37-4673-43af-9963-9c3c393fd037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:20.306218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-801d676f-56a5-4e88-8de2-a061583486f7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:20.308476 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-801d676f-56a5-4e88-8de2-a061583486f7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:20.321457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-801d676f-56a5-4e88-8de2-a061583486f7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:20.322558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-801d676f-56a5-4e88-8de2-a061583486f7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:20.323444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-801d676f-56a5-4e88-8de2-a061583486f7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:20.329843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-801d676f-56a5-4e88-8de2-a061583486f7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:20.330589 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-801d676f-56a5-4e88-8de2-a061583486f7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:20.331263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-801d676f-56a5-4e88-8de2-a061583486f7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:20.417193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7af73e37-4673-43af-9963-9c3c393fd037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:20.433310 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7af73e37-4673-43af-9963-9c3c393fd037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1192 microversion: 2.1 time: 0.190009 Aug 07 01:14:20.434518 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 281/538] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:20 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1192 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:20.544300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-801d676f-56a5-4e88-8de2-a061583486f7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:20.555479 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-801d676f-56a5-4e88-8de2-a061583486f7 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.267408 Aug 07 01:14:20.556714 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 258/539] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:20 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:20.976853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-716e203a-7c62-40f0-8748-5c1f18f881b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:20.977653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-716e203a-7c62-40f0-8748-5c1f18f881b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:20.985082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-716e203a-7c62-40f0-8748-5c1f18f881b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:20.986370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-716e203a-7c62-40f0-8748-5c1f18f881b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:20.986912 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-716e203a-7c62-40f0-8748-5c1f18f881b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:20.987880 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-716e203a-7c62-40f0-8748-5c1f18f881b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:20.988460 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-716e203a-7c62-40f0-8748-5c1f18f881b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:20.988966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-716e203a-7c62-40f0-8748-5c1f18f881b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:21.313119 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-716e203a-7c62-40f0-8748-5c1f18f881b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:21.341682 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-716e203a-7c62-40f0-8748-5c1f18f881b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1565 microversion: 2.1 time: 0.368109 Aug 07 01:14:21.344567 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 282/540] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:20 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1565 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:21.376879 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e23374a3-32c0-4c8a-b5e6-9712e6426752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:21.378082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e23374a3-32c0-4c8a-b5e6-9712e6426752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:21.385596 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e23374a3-32c0-4c8a-b5e6-9712e6426752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:21.386417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e23374a3-32c0-4c8a-b5e6-9712e6426752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:21.387075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e23374a3-32c0-4c8a-b5e6-9712e6426752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:21.388160 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e23374a3-32c0-4c8a-b5e6-9712e6426752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:21.389209 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e23374a3-32c0-4c8a-b5e6-9712e6426752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:21.390055 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e23374a3-32c0-4c8a-b5e6-9712e6426752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:21.458798 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2a469eec-bc6d-4b4c-8160-81812abaf930 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:21.459189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2a469eec-bc6d-4b4c-8160-81812abaf930 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:21.466871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2a469eec-bc6d-4b4c-8160-81812abaf930 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:21.467470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2a469eec-bc6d-4b4c-8160-81812abaf930 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:21.468619 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2a469eec-bc6d-4b4c-8160-81812abaf930 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:21.469443 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2a469eec-bc6d-4b4c-8160-81812abaf930 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:21.470520 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2a469eec-bc6d-4b4c-8160-81812abaf930 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:21.471053 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2a469eec-bc6d-4b4c-8160-81812abaf930 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:21.577426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e23374a3-32c0-4c8a-b5e6-9712e6426752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.204921 Aug 07 01:14:21.578639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 259/541] 10.222.0.26 () {58 vars in 1275 bytes} [Sun Aug 7 01:14:21 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-interface => generated 285 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:14:21.597808 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a97f39cd-ce4b-496d-9f5f-f8f80676e92b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:21.599356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2a469eec-bc6d-4b4c-8160-81812abaf930 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:21.600138 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a97f39cd-ce4b-496d-9f5f-f8f80676e92b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:21.615190 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2a469eec-bc6d-4b4c-8160-81812abaf930 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1210 microversion: 2.1 time: 0.156278 Aug 07 01:14:21.615573 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 283/542] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:21 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1210 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:21.620364 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a97f39cd-ce4b-496d-9f5f-f8f80676e92b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:21.621816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a97f39cd-ce4b-496d-9f5f-f8f80676e92b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:21.622897 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a97f39cd-ce4b-496d-9f5f-f8f80676e92b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:21.627092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a97f39cd-ce4b-496d-9f5f-f8f80676e92b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:21.628483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a97f39cd-ce4b-496d-9f5f-f8f80676e92b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:21.629620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a97f39cd-ce4b-496d-9f5f-f8f80676e92b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:21.975618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a97f39cd-ce4b-496d-9f5f-f8f80676e92b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:22.000370 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a97f39cd-ce4b-496d-9f5f-f8f80676e92b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.411504 Aug 07 01:14:22.024690 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 260/543] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:21 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:22.168723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-fd1dfcb3-7678-4e5e-b058-fee1897a68b5 req-53cee0ee-27e2-462d-b2fc-1857ee3b07e6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e86efc1a-b689-454a-8683-526a9c18e222", "tag": "f5f83050-c635-4216-9f73-3f7cd60024ff"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:22.182906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-fd1dfcb3-7678-4e5e-b058-fee1897a68b5 req-53cee0ee-27e2-462d-b2fc-1857ee3b07e6 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:22.182906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-fd1dfcb3-7678-4e5e-b058-fee1897a68b5 req-53cee0ee-27e2-462d-b2fc-1857ee3b07e6 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:22.182906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-fd1dfcb3-7678-4e5e-b058-fee1897a68b5 req-53cee0ee-27e2-462d-b2fc-1857ee3b07e6 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:22.198879 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-fd1dfcb3-7678-4e5e-b058-fee1897a68b5 req-53cee0ee-27e2-462d-b2fc-1857ee3b07e6 service nova] Creating event network-changed:f5f83050-c635-4216-9f73-3f7cd60024ff for instance e86efc1a-b689-454a-8683-526a9c18e222 on hv-cinder-maste Aug 07 01:14:22.206925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-fd1dfcb3-7678-4e5e-b058-fee1897a68b5 req-53cee0ee-27e2-462d-b2fc-1857ee3b07e6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042540 Aug 07 01:14:22.207873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 284/544] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:14:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:14:22.649123 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f7392118-96f5-4f0d-956a-480f770d45ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:22.650301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f7392118-96f5-4f0d-956a-480f770d45ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:22.660337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f7392118-96f5-4f0d-956a-480f770d45ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:22.661374 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f7392118-96f5-4f0d-956a-480f770d45ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:22.662099 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f7392118-96f5-4f0d-956a-480f770d45ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:22.663383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f7392118-96f5-4f0d-956a-480f770d45ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:22.663880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f7392118-96f5-4f0d-956a-480f770d45ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:22.666052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f7392118-96f5-4f0d-956a-480f770d45ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:23.046475 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f7392118-96f5-4f0d-956a-480f770d45ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:23.060661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-33be4976-8727-49de-b211-49834c48c567 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:23.070339 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-33be4976-8727-49de-b211-49834c48c567 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:23.089394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f7392118-96f5-4f0d-956a-480f770d45ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1278 microversion: 2.1 time: 0.443053 Aug 07 01:14:23.091772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 261/545] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:22 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1278 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:23.139044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-33be4976-8727-49de-b211-49834c48c567 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:23.140562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-33be4976-8727-49de-b211-49834c48c567 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:23.144494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-33be4976-8727-49de-b211-49834c48c567 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:23.146549 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-33be4976-8727-49de-b211-49834c48c567 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:23.147919 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-33be4976-8727-49de-b211-49834c48c567 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:23.149234 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-33be4976-8727-49de-b211-49834c48c567 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:23.552107 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-3f4ce20e-9559-439d-b522-1f23914e9f84 req-b1273495-2aea-40a1-8bac-1cdb4cfc77d4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9968bbaa-9908-4063-bc7f-e6e805385fa2", "tag": "6681d8c9-7955-47db-a880-4ab2cb9ac6d1"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:23.581390 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-3f4ce20e-9559-439d-b522-1f23914e9f84 req-b1273495-2aea-40a1-8bac-1cdb4cfc77d4 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:23.583048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-3f4ce20e-9559-439d-b522-1f23914e9f84 req-b1273495-2aea-40a1-8bac-1cdb4cfc77d4 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:23.584440 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-3f4ce20e-9559-439d-b522-1f23914e9f84 req-b1273495-2aea-40a1-8bac-1cdb4cfc77d4 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:23.614065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-3f4ce20e-9559-439d-b522-1f23914e9f84 req-b1273495-2aea-40a1-8bac-1cdb4cfc77d4 service nova] Creating event network-changed:6681d8c9-7955-47db-a880-4ab2cb9ac6d1 for instance 9968bbaa-9908-4063-bc7f-e6e805385fa2 on hv-cinder-maste Aug 07 01:14:23.625933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-3f4ce20e-9559-439d-b522-1f23914e9f84 req-b1273495-2aea-40a1-8bac-1cdb4cfc77d4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.084371 Aug 07 01:14:23.628862 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 262/546] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:14:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:14:23.949596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-33be4976-8727-49de-b211-49834c48c567 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:23.967270 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-33be4976-8727-49de-b211-49834c48c567 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.931423 Aug 07 01:14:23.968341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 285/547] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:23 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 934 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:24.136192 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e0791ee9-5d05-4198-b5f2-63d1cd3d53dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:24.138255 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e0791ee9-5d05-4198-b5f2-63d1cd3d53dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:24.147933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0791ee9-5d05-4198-b5f2-63d1cd3d53dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:24.148566 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0791ee9-5d05-4198-b5f2-63d1cd3d53dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:24.149052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0791ee9-5d05-4198-b5f2-63d1cd3d53dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:24.149946 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0791ee9-5d05-4198-b5f2-63d1cd3d53dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:24.150407 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0791ee9-5d05-4198-b5f2-63d1cd3d53dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:24.151237 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e0791ee9-5d05-4198-b5f2-63d1cd3d53dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:24.362782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e0791ee9-5d05-4198-b5f2-63d1cd3d53dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:24.373909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-720e9adc-79ff-40e9-a4a6-6a8b9793df56 req-de8348a8-55a7-4983-a1ef-44d764e9a21d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6ac28640-d6da-4cfc-b7ad-0d69136ac1fd", "tag": "4d7d0711-c6ef-4ae2-b272-c2cf175993d4"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:24.379055 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e0791ee9-5d05-4198-b5f2-63d1cd3d53dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.251244 Aug 07 01:14:24.380731 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 263/548] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:24 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:24.398873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-720e9adc-79ff-40e9-a4a6-6a8b9793df56 req-de8348a8-55a7-4983-a1ef-44d764e9a21d service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:24.400429 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-720e9adc-79ff-40e9-a4a6-6a8b9793df56 req-de8348a8-55a7-4983-a1ef-44d764e9a21d service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:24.401302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-720e9adc-79ff-40e9-a4a6-6a8b9793df56 req-de8348a8-55a7-4983-a1ef-44d764e9a21d service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:24.428478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-720e9adc-79ff-40e9-a4a6-6a8b9793df56 req-de8348a8-55a7-4983-a1ef-44d764e9a21d service nova] Creating event network-changed:4d7d0711-c6ef-4ae2-b272-c2cf175993d4 for instance 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd on hv-cinder-maste Aug 07 01:14:24.438306 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-720e9adc-79ff-40e9-a4a6-6a8b9793df56 req-de8348a8-55a7-4983-a1ef-44d764e9a21d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068097 Aug 07 01:14:24.439626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 286/549] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:14:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:14:24.999511 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-db0beac1-35ca-4018-8181-81406c1f4956 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:25.000514 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-db0beac1-35ca-4018-8181-81406c1f4956 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:25.008900 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-db0beac1-35ca-4018-8181-81406c1f4956 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:25.009494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-db0beac1-35ca-4018-8181-81406c1f4956 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:25.010054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-db0beac1-35ca-4018-8181-81406c1f4956 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:25.011058 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-db0beac1-35ca-4018-8181-81406c1f4956 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:25.011798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-db0beac1-35ca-4018-8181-81406c1f4956 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:25.012321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-db0beac1-35ca-4018-8181-81406c1f4956 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:25.209706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-db0beac1-35ca-4018-8181-81406c1f4956 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:25.221433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-db0beac1-35ca-4018-8181-81406c1f4956 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.226396 Aug 07 01:14:25.222460 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 264/550] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:24 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:25.410306 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-381d23bc-f271-4e43-9f50-c9c208328ee3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:25.411246 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-381d23bc-f271-4e43-9f50-c9c208328ee3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:25.418633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-381d23bc-f271-4e43-9f50-c9c208328ee3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:25.419156 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-381d23bc-f271-4e43-9f50-c9c208328ee3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:25.419602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-381d23bc-f271-4e43-9f50-c9c208328ee3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:25.420393 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-381d23bc-f271-4e43-9f50-c9c208328ee3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:25.420967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-381d23bc-f271-4e43-9f50-c9c208328ee3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:25.421403 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-381d23bc-f271-4e43-9f50-c9c208328ee3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:25.642013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-381d23bc-f271-4e43-9f50-c9c208328ee3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:25.653289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-381d23bc-f271-4e43-9f50-c9c208328ee3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.248295 Aug 07 01:14:25.654137 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 287/551] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:25 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:26.249676 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e41d689d-1e45-465d-84d7-ae1abbea98e8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:26.250784 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e41d689d-1e45-465d-84d7-ae1abbea98e8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:26.260835 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e41d689d-1e45-465d-84d7-ae1abbea98e8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:26.261404 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e41d689d-1e45-465d-84d7-ae1abbea98e8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:26.261888 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e41d689d-1e45-465d-84d7-ae1abbea98e8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:26.262783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e41d689d-1e45-465d-84d7-ae1abbea98e8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:26.263235 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e41d689d-1e45-465d-84d7-ae1abbea98e8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:26.263678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e41d689d-1e45-465d-84d7-ae1abbea98e8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:26.487113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e41d689d-1e45-465d-84d7-ae1abbea98e8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:26.500129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e41d689d-1e45-465d-84d7-ae1abbea98e8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.256531 Aug 07 01:14:26.501932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 265/552] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:26 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:26.683917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-69252c73-e4ce-4ff1-8804-19f896f8f6b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:26.684632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-69252c73-e4ce-4ff1-8804-19f896f8f6b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:26.693381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69252c73-e4ce-4ff1-8804-19f896f8f6b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:26.694033 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69252c73-e4ce-4ff1-8804-19f896f8f6b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:26.694587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69252c73-e4ce-4ff1-8804-19f896f8f6b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:26.695442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69252c73-e4ce-4ff1-8804-19f896f8f6b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:26.696019 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69252c73-e4ce-4ff1-8804-19f896f8f6b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:26.696613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69252c73-e4ce-4ff1-8804-19f896f8f6b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:26.883987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-69252c73-e4ce-4ff1-8804-19f896f8f6b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:26.898295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-69252c73-e4ce-4ff1-8804-19f896f8f6b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.218679 Aug 07 01:14:26.899330 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 288/553] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:26 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:27.540212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-377659bb-2b3d-4a85-ab5c-0b2c3daa2cb8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:27.541449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-377659bb-2b3d-4a85-ab5c-0b2c3daa2cb8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:27.545068 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=4 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:27.546298 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=16 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:27.558436 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-377659bb-2b3d-4a85-ab5c-0b2c3daa2cb8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:27.559433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-377659bb-2b3d-4a85-ab5c-0b2c3daa2cb8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:27.560280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-377659bb-2b3d-4a85-ab5c-0b2c3daa2cb8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:27.561695 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-377659bb-2b3d-4a85-ab5c-0b2c3daa2cb8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:27.562542 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-377659bb-2b3d-4a85-ab5c-0b2c3daa2cb8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:27.563321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-377659bb-2b3d-4a85-ab5c-0b2c3daa2cb8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:27.774414 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-377659bb-2b3d-4a85-ab5c-0b2c3daa2cb8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:27.788514 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-377659bb-2b3d-4a85-ab5c-0b2c3daa2cb8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.255580 Aug 07 01:14:27.789443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 266/554] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:27 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:27.927716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f6db6990-c38d-45b4-9d8f-211d60a47dd8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:27.928501 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f6db6990-c38d-45b4-9d8f-211d60a47dd8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:27.930696 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:27.939887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f6db6990-c38d-45b4-9d8f-211d60a47dd8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:27.940230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f6db6990-c38d-45b4-9d8f-211d60a47dd8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:27.941034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f6db6990-c38d-45b4-9d8f-211d60a47dd8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:27.941897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f6db6990-c38d-45b4-9d8f-211d60a47dd8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:27.942472 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f6db6990-c38d-45b4-9d8f-211d60a47dd8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:27.943096 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f6db6990-c38d-45b4-9d8f-211d60a47dd8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:28.000745 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=12 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:28.172116 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f6db6990-c38d-45b4-9d8f-211d60a47dd8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:28.186867 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f6db6990-c38d-45b4-9d8f-211d60a47dd8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.264990 Aug 07 01:14:28.187956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 289/555] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:27 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:28.824602 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-00e3f9ad-0ec8-49fb-ae6f-c4c05758ce89 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:28.825504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-00e3f9ad-0ec8-49fb-ae6f-c4c05758ce89 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:28.833570 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00e3f9ad-0ec8-49fb-ae6f-c4c05758ce89 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:28.834869 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00e3f9ad-0ec8-49fb-ae6f-c4c05758ce89 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:28.835354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00e3f9ad-0ec8-49fb-ae6f-c4c05758ce89 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:28.836087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00e3f9ad-0ec8-49fb-ae6f-c4c05758ce89 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:28.836582 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00e3f9ad-0ec8-49fb-ae6f-c4c05758ce89 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:28.837008 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00e3f9ad-0ec8-49fb-ae6f-c4c05758ce89 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:29.029334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-00e3f9ad-0ec8-49fb-ae6f-c4c05758ce89 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:29.043827 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-00e3f9ad-0ec8-49fb-ae6f-c4c05758ce89 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.223502 Aug 07 01:14:29.044947 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 267/556] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:28 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:29.222607 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9e9ee65a-db9c-4668-ae6f-a6e991536300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:29.223453 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9e9ee65a-db9c-4668-ae6f-a6e991536300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:29.231000 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e9ee65a-db9c-4668-ae6f-a6e991536300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:29.231519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e9ee65a-db9c-4668-ae6f-a6e991536300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:29.231976 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e9ee65a-db9c-4668-ae6f-a6e991536300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:29.232876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e9ee65a-db9c-4668-ae6f-a6e991536300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:29.233338 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e9ee65a-db9c-4668-ae6f-a6e991536300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:29.233771 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e9ee65a-db9c-4668-ae6f-a6e991536300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:29.407417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9e9ee65a-db9c-4668-ae6f-a6e991536300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:29.415989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9e9ee65a-db9c-4668-ae6f-a6e991536300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.198478 Aug 07 01:14:29.416635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 290/557] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:29 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:30.081859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f0880957-f800-46f3-9991-099df5117488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:30.082601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f0880957-f800-46f3-9991-099df5117488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:30.090396 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0880957-f800-46f3-9991-099df5117488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:30.090928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0880957-f800-46f3-9991-099df5117488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:30.091473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0880957-f800-46f3-9991-099df5117488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:30.092408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0880957-f800-46f3-9991-099df5117488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:30.092904 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0880957-f800-46f3-9991-099df5117488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:30.093372 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0880957-f800-46f3-9991-099df5117488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:30.243599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f0880957-f800-46f3-9991-099df5117488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:30.253029 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f0880957-f800-46f3-9991-099df5117488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.175444 Aug 07 01:14:30.253800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 268/558] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:30 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:30.438493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9729fab4-50ba-4590-a33e-c5412240f399 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:30.439167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9729fab4-50ba-4590-a33e-c5412240f399 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:30.446209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9729fab4-50ba-4590-a33e-c5412240f399 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:30.446704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9729fab4-50ba-4590-a33e-c5412240f399 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:30.447157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9729fab4-50ba-4590-a33e-c5412240f399 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:30.448136 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9729fab4-50ba-4590-a33e-c5412240f399 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:30.448632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9729fab4-50ba-4590-a33e-c5412240f399 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:30.449059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9729fab4-50ba-4590-a33e-c5412240f399 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:30.609857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9729fab4-50ba-4590-a33e-c5412240f399 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:30.618624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9729fab4-50ba-4590-a33e-c5412240f399 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.184139 Aug 07 01:14:30.619308 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 291/559] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:30 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:31.279076 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1f90a919-897f-4476-9fb7-9fad08b8ed36 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:31.280600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1f90a919-897f-4476-9fb7-9fad08b8ed36 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:31.292770 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f90a919-897f-4476-9fb7-9fad08b8ed36 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:31.293662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f90a919-897f-4476-9fb7-9fad08b8ed36 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:31.294435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f90a919-897f-4476-9fb7-9fad08b8ed36 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:31.295693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f90a919-897f-4476-9fb7-9fad08b8ed36 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:31.296565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f90a919-897f-4476-9fb7-9fad08b8ed36 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:31.297349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f90a919-897f-4476-9fb7-9fad08b8ed36 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:31.526606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1f90a919-897f-4476-9fb7-9fad08b8ed36 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:31.541165 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1f90a919-897f-4476-9fb7-9fad08b8ed36 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.268986 Aug 07 01:14:31.542237 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 269/560] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:31 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:31.639849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7fd28053-ff1e-4e6c-9b8f-e86509420f82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:31.640849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7fd28053-ff1e-4e6c-9b8f-e86509420f82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:31.648774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7fd28053-ff1e-4e6c-9b8f-e86509420f82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:31.649290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7fd28053-ff1e-4e6c-9b8f-e86509420f82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:31.649786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7fd28053-ff1e-4e6c-9b8f-e86509420f82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:31.650626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7fd28053-ff1e-4e6c-9b8f-e86509420f82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:31.651356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7fd28053-ff1e-4e6c-9b8f-e86509420f82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:31.652084 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7fd28053-ff1e-4e6c-9b8f-e86509420f82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:31.854221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7fd28053-ff1e-4e6c-9b8f-e86509420f82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:31.863724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7fd28053-ff1e-4e6c-9b8f-e86509420f82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.228152 Aug 07 01:14:31.864530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 292/561] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:31 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:32.583352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8f755440-6c94-4751-8c5f-fc020f25727e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:32.584702 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8f755440-6c94-4751-8c5f-fc020f25727e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:32.597452 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f755440-6c94-4751-8c5f-fc020f25727e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:32.598720 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f755440-6c94-4751-8c5f-fc020f25727e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:32.599705 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f755440-6c94-4751-8c5f-fc020f25727e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:32.601310 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f755440-6c94-4751-8c5f-fc020f25727e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:32.602215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f755440-6c94-4751-8c5f-fc020f25727e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:32.603077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8f755440-6c94-4751-8c5f-fc020f25727e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:32.916435 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cdc8af58-683c-4ca9-b8ce-6f5c6baeb3cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:32.920471 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cdc8af58-683c-4ca9-b8ce-6f5c6baeb3cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:32.949071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cdc8af58-683c-4ca9-b8ce-6f5c6baeb3cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:32.952393 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cdc8af58-683c-4ca9-b8ce-6f5c6baeb3cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:32.953792 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cdc8af58-683c-4ca9-b8ce-6f5c6baeb3cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:32.956796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cdc8af58-683c-4ca9-b8ce-6f5c6baeb3cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:32.958128 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cdc8af58-683c-4ca9-b8ce-6f5c6baeb3cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:32.959387 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cdc8af58-683c-4ca9-b8ce-6f5c6baeb3cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:33.025931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8f755440-6c94-4751-8c5f-fc020f25727e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:33.044970 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8f755440-6c94-4751-8c5f-fc020f25727e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.469583 Aug 07 01:14:33.059691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 270/562] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:32 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:33.248150 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-cdc8af58-683c-4ca9-b8ce-6f5c6baeb3cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:33.263425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cdc8af58-683c-4ca9-b8ce-6f5c6baeb3cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.363934 Aug 07 01:14:33.264733 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 293/563] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:32 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:34.083971 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4fc43f7e-7853-4b7c-9873-8261452472c2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:34.084895 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4fc43f7e-7853-4b7c-9873-8261452472c2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:34.093536 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4fc43f7e-7853-4b7c-9873-8261452472c2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:34.094145 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4fc43f7e-7853-4b7c-9873-8261452472c2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:34.094628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4fc43f7e-7853-4b7c-9873-8261452472c2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:34.095935 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4fc43f7e-7853-4b7c-9873-8261452472c2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:34.096624 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4fc43f7e-7853-4b7c-9873-8261452472c2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:34.097114 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4fc43f7e-7853-4b7c-9873-8261452472c2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:34.292391 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-acde2f3e-2449-4748-a1c9-aad00edefd3c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:34.294694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-acde2f3e-2449-4748-a1c9-aad00edefd3c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:34.307975 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acde2f3e-2449-4748-a1c9-aad00edefd3c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:34.308945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acde2f3e-2449-4748-a1c9-aad00edefd3c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:34.309781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acde2f3e-2449-4748-a1c9-aad00edefd3c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:34.310563 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4fc43f7e-7853-4b7c-9873-8261452472c2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:34.311136 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acde2f3e-2449-4748-a1c9-aad00edefd3c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:34.311987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acde2f3e-2449-4748-a1c9-aad00edefd3c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:34.312829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acde2f3e-2449-4748-a1c9-aad00edefd3c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:34.324389 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4fc43f7e-7853-4b7c-9873-8261452472c2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.245427 Aug 07 01:14:34.325631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 271/564] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:34 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:34.537525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-acde2f3e-2449-4748-a1c9-aad00edefd3c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:34.546820 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-acde2f3e-2449-4748-a1c9-aad00edefd3c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.262309 Aug 07 01:14:34.547677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 294/565] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:34 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:35.359913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-85bc9948-a2b8-4026-bd2b-40596e52fac3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:35.361618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-85bc9948-a2b8-4026-bd2b-40596e52fac3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:35.370149 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-85bc9948-a2b8-4026-bd2b-40596e52fac3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:35.370821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-85bc9948-a2b8-4026-bd2b-40596e52fac3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:35.371450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-85bc9948-a2b8-4026-bd2b-40596e52fac3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:35.372402 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-85bc9948-a2b8-4026-bd2b-40596e52fac3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:35.372988 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-85bc9948-a2b8-4026-bd2b-40596e52fac3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:35.373521 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-85bc9948-a2b8-4026-bd2b-40596e52fac3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:35.535678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-85bc9948-a2b8-4026-bd2b-40596e52fac3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:35.550813 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-85bc9948-a2b8-4026-bd2b-40596e52fac3 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.195073 Aug 07 01:14:35.551882 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 272/566] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:35 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:35.575067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5b639f11-bb51-4f7f-8bce-ff0b4a4b015d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:35.576299 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5b639f11-bb51-4f7f-8bce-ff0b4a4b015d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:35.588387 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b639f11-bb51-4f7f-8bce-ff0b4a4b015d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:35.589263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b639f11-bb51-4f7f-8bce-ff0b4a4b015d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:35.590085 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b639f11-bb51-4f7f-8bce-ff0b4a4b015d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:35.591433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b639f11-bb51-4f7f-8bce-ff0b4a4b015d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:35.592286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b639f11-bb51-4f7f-8bce-ff0b4a4b015d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:35.593046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b639f11-bb51-4f7f-8bce-ff0b4a4b015d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:35.601965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:35.817256 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5b639f11-bb51-4f7f-8bce-ff0b4a4b015d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:35.828023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5b639f11-bb51-4f7f-8bce-ff0b4a4b015d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.260338 Aug 07 01:14:35.829733 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 295/567] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:35 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:36.465772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1dd5ed10-8f4e-4a88-bfce-ba8539ad6b2e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:36.466560 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1dd5ed10-8f4e-4a88-bfce-ba8539ad6b2e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:36.474473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1dd5ed10-8f4e-4a88-bfce-ba8539ad6b2e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:36.474967 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1dd5ed10-8f4e-4a88-bfce-ba8539ad6b2e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:36.475449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1dd5ed10-8f4e-4a88-bfce-ba8539ad6b2e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:36.476180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1dd5ed10-8f4e-4a88-bfce-ba8539ad6b2e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:36.476774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1dd5ed10-8f4e-4a88-bfce-ba8539ad6b2e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:36.477210 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1dd5ed10-8f4e-4a88-bfce-ba8539ad6b2e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:36.512711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1dd5ed10-8f4e-4a88-bfce-ba8539ad6b2e tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/os-volume_attachments/9bffc979-92b9-42b4-a2f9-1d3f5dd0e21c" status: 200 len: 194 microversion: 2.20 time: 0.051244 Aug 07 01:14:36.513244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 273/568] 10.222.0.26 () {60 vars in 1454 bytes} [Sun Aug 7 01:14:36 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/os-volume_attachments/9bffc979-92b9-42b4-a2f9-1d3f5dd0e21c => generated 194 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:14:36.588710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3a50bc75-b568-4c2e-8dad-3d57bf8a2cdf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:36.589663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3a50bc75-b568-4c2e-8dad-3d57bf8a2cdf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:36.601916 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a50bc75-b568-4c2e-8dad-3d57bf8a2cdf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:36.602795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a50bc75-b568-4c2e-8dad-3d57bf8a2cdf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:36.603539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a50bc75-b568-4c2e-8dad-3d57bf8a2cdf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:36.603838 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c4d80adc-8797-469a-b30d-ff7290e80638 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:36.604644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c4d80adc-8797-469a-b30d-ff7290e80638 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:36.605816 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a50bc75-b568-4c2e-8dad-3d57bf8a2cdf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:36.606728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a50bc75-b568-4c2e-8dad-3d57bf8a2cdf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:36.607451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a50bc75-b568-4c2e-8dad-3d57bf8a2cdf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:36.613858 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4d80adc-8797-469a-b30d-ff7290e80638 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:36.614501 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4d80adc-8797-469a-b30d-ff7290e80638 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:36.615119 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4d80adc-8797-469a-b30d-ff7290e80638 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:36.616320 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4d80adc-8797-469a-b30d-ff7290e80638 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:36.616907 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4d80adc-8797-469a-b30d-ff7290e80638 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:36.617579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4d80adc-8797-469a-b30d-ff7290e80638 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:36.728812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:36.740838 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.objects.instance [None req-c4d80adc-8797-469a-b30d-ff7290e80638 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lazy-loading 'services' on Instance uuid 7f60fb22-d17a-4b7c-9b7d-95543be1907e {{(pid=117984) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:14:36.852833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3a50bc75-b568-4c2e-8dad-3d57bf8a2cdf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:36.872656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3a50bc75-b568-4c2e-8dad-3d57bf8a2cdf tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.288028 Aug 07 01:14:36.874031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 296/569] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:36 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:36.883867 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-235279cf-ad7e-42da-a76d-be9fd29b8a47 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:36.885786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-235279cf-ad7e-42da-a76d-be9fd29b8a47 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:36.899567 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-235279cf-ad7e-42da-a76d-be9fd29b8a47 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:36.901537 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-235279cf-ad7e-42da-a76d-be9fd29b8a47 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:36.901895 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c4d80adc-8797-469a-b30d-ff7290e80638 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "DELETE /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/os-volume_attachments/9bffc979-92b9-42b4-a2f9-1d3f5dd0e21c" status: 202 len: 0 microversion: 2.20 time: 0.302121 Aug 07 01:14:36.902406 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-235279cf-ad7e-42da-a76d-be9fd29b8a47 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:36.903540 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 274/570] 10.222.0.26 () {60 vars in 1457 bytes} [Sun Aug 7 01:14:36 2022] DELETE /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/os-volume_attachments/9bffc979-92b9-42b4-a2f9-1d3f5dd0e21c => generated 0 bytes in 305 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Aug 07 01:14:36.903839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-235279cf-ad7e-42da-a76d-be9fd29b8a47 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:36.904820 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-235279cf-ad7e-42da-a76d-be9fd29b8a47 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:36.905705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-235279cf-ad7e-42da-a76d-be9fd29b8a47 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:36.918313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9467a9ba-b49e-407b-978c-13e137cf9b5a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:36.921329 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9467a9ba-b49e-407b-978c-13e137cf9b5a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:36.932569 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9467a9ba-b49e-407b-978c-13e137cf9b5a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:36.933073 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9467a9ba-b49e-407b-978c-13e137cf9b5a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:36.933509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9467a9ba-b49e-407b-978c-13e137cf9b5a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:36.934236 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9467a9ba-b49e-407b-978c-13e137cf9b5a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:36.934700 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9467a9ba-b49e-407b-978c-13e137cf9b5a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:36.936006 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9467a9ba-b49e-407b-978c-13e137cf9b5a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:36.979384 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9467a9ba-b49e-407b-978c-13e137cf9b5a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.066861 Aug 07 01:14:36.986279 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 275/571] 10.222.0.26 () {60 vars in 1343 bytes} [Sun Aug 7 01:14:36 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 07 01:14:37.100598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-235279cf-ad7e-42da-a76d-be9fd29b8a47 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:37.109015 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-235279cf-ad7e-42da-a76d-be9fd29b8a47 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.233034 Aug 07 01:14:37.109610 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 297/572] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:36 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:37.898322 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-15fe0eca-2cae-4503-a284-c7150d189954 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:37.898968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-15fe0eca-2cae-4503-a284-c7150d189954 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:37.906782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15fe0eca-2cae-4503-a284-c7150d189954 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:37.907475 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15fe0eca-2cae-4503-a284-c7150d189954 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:37.907929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15fe0eca-2cae-4503-a284-c7150d189954 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:37.908825 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15fe0eca-2cae-4503-a284-c7150d189954 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:37.909367 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15fe0eca-2cae-4503-a284-c7150d189954 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:37.909797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15fe0eca-2cae-4503-a284-c7150d189954 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:37.996395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-378c8bd7-9752-46bc-a264-75b589682628 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:37.999487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-378c8bd7-9752-46bc-a264-75b589682628 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:38.009080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-378c8bd7-9752-46bc-a264-75b589682628 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:38.009855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-378c8bd7-9752-46bc-a264-75b589682628 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:38.010572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-378c8bd7-9752-46bc-a264-75b589682628 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:38.011774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-378c8bd7-9752-46bc-a264-75b589682628 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:38.012622 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-378c8bd7-9752-46bc-a264-75b589682628 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:38.013359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-378c8bd7-9752-46bc-a264-75b589682628 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:38.051140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-378c8bd7-9752-46bc-a264-75b589682628 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.060055 Aug 07 01:14:38.051798 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 298/573] 10.222.0.26 () {60 vars in 1343 bytes} [Sun Aug 7 01:14:37 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 07 01:14:38.085496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-15fe0eca-2cae-4503-a284-c7150d189954 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:38.094571 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-15fe0eca-2cae-4503-a284-c7150d189954 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.200152 Aug 07 01:14:38.095738 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 276/574] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:37 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:38.133183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5f39d1da-2052-4036-93fe-8d221cb165d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:38.133946 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5f39d1da-2052-4036-93fe-8d221cb165d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:38.141721 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5f39d1da-2052-4036-93fe-8d221cb165d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:38.142380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5f39d1da-2052-4036-93fe-8d221cb165d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:38.142930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5f39d1da-2052-4036-93fe-8d221cb165d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:38.143797 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5f39d1da-2052-4036-93fe-8d221cb165d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:38.144409 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5f39d1da-2052-4036-93fe-8d221cb165d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:38.144953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5f39d1da-2052-4036-93fe-8d221cb165d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:38.347065 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5f39d1da-2052-4036-93fe-8d221cb165d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:38.357479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5f39d1da-2052-4036-93fe-8d221cb165d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.228543 Aug 07 01:14:38.358270 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 299/575] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:38 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:38.736911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-867604b0-9776-401a-99cc-a4cc61cc7095 req-b777ccfd-b473-4d63-810f-ee4348c441ed service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e86efc1a-b689-454a-8683-526a9c18e222", "name": "network-vif-plugged", "status": "completed", "tag": "f5f83050-c635-4216-9f73-3f7cd60024ff"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:38.751562 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-867604b0-9776-401a-99cc-a4cc61cc7095 req-b777ccfd-b473-4d63-810f-ee4348c441ed service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:38.752390 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-867604b0-9776-401a-99cc-a4cc61cc7095 req-b777ccfd-b473-4d63-810f-ee4348c441ed service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:38.752965 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-867604b0-9776-401a-99cc-a4cc61cc7095 req-b777ccfd-b473-4d63-810f-ee4348c441ed service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:38.773961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-867604b0-9776-401a-99cc-a4cc61cc7095 req-b777ccfd-b473-4d63-810f-ee4348c441ed service nova] Creating event network-vif-plugged:f5f83050-c635-4216-9f73-3f7cd60024ff for instance e86efc1a-b689-454a-8683-526a9c18e222 on hv-cinder-maste Aug 07 01:14:38.780641 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-867604b0-9776-401a-99cc-a4cc61cc7095 req-b777ccfd-b473-4d63-810f-ee4348c441ed service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052350 Aug 07 01:14:38.781405 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 277/576] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:14:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:14:39.070253 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f9174a42-ed97-4f53-bc3a-b14c819f15c5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:39.074727 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f9174a42-ed97-4f53-bc3a-b14c819f15c5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:39.088853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9174a42-ed97-4f53-bc3a-b14c819f15c5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:39.089780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9174a42-ed97-4f53-bc3a-b14c819f15c5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:39.090724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9174a42-ed97-4f53-bc3a-b14c819f15c5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:39.092080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9174a42-ed97-4f53-bc3a-b14c819f15c5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:39.093047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9174a42-ed97-4f53-bc3a-b14c819f15c5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:39.093942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9174a42-ed97-4f53-bc3a-b14c819f15c5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:39.123853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fbe027e1-19d3-4eb9-94a3-d2991cbfa051 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:39.125226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fbe027e1-19d3-4eb9-94a3-d2991cbfa051 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:39.138079 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbe027e1-19d3-4eb9-94a3-d2991cbfa051 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:39.138687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbe027e1-19d3-4eb9-94a3-d2991cbfa051 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:39.139135 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbe027e1-19d3-4eb9-94a3-d2991cbfa051 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:39.139857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbe027e1-19d3-4eb9-94a3-d2991cbfa051 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:39.140341 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbe027e1-19d3-4eb9-94a3-d2991cbfa051 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:39.140773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbe027e1-19d3-4eb9-94a3-d2991cbfa051 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:39.155730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f9174a42-ed97-4f53-bc3a-b14c819f15c5 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.093434 Aug 07 01:14:39.157139 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 300/577] 10.222.0.26 () {60 vars in 1343 bytes} [Sun Aug 7 01:14:39 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e/os-volume_attachments => generated 25 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:39.277536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ec39a9cf-ce35-46ac-869d-4e6c7d55fdd6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:39.278201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ec39a9cf-ce35-46ac-869d-4e6c7d55fdd6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:39.292002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec39a9cf-ce35-46ac-869d-4e6c7d55fdd6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:39.293461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec39a9cf-ce35-46ac-869d-4e6c7d55fdd6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:39.295226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec39a9cf-ce35-46ac-869d-4e6c7d55fdd6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:39.296709 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec39a9cf-ce35-46ac-869d-4e6c7d55fdd6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:39.297574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec39a9cf-ce35-46ac-869d-4e6c7d55fdd6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:39.298670 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec39a9cf-ce35-46ac-869d-4e6c7d55fdd6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:39.350808 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fbe027e1-19d3-4eb9-94a3-d2991cbfa051 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:39.360390 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ec39a9cf-ce35-46ac-869d-4e6c7d55fdd6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Going to try to terminate instance {{(pid=117983) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:14:39.369874 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fbe027e1-19d3-4eb9-94a3-d2991cbfa051 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.250873 Aug 07 01:14:39.382052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 278/578] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:39 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:39.389075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-693e41ba-dadf-46ac-9279-db9f8ccea300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:39.389891 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-693e41ba-dadf-46ac-9279-db9f8ccea300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:39.402066 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-693e41ba-dadf-46ac-9279-db9f8ccea300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:39.402781 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-693e41ba-dadf-46ac-9279-db9f8ccea300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:39.403362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-693e41ba-dadf-46ac-9279-db9f8ccea300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:39.405131 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-693e41ba-dadf-46ac-9279-db9f8ccea300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:39.405764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-693e41ba-dadf-46ac-9279-db9f8ccea300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:39.406473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-693e41ba-dadf-46ac-9279-db9f8ccea300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:39.521979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ec39a9cf-ce35-46ac-869d-4e6c7d55fdd6 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "DELETE /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 204 len: 0 microversion: 2.20 time: 0.251525 Aug 07 01:14:39.522831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 301/579] 10.222.0.26 () {60 vars in 1280 bytes} [Sun Aug 7 01:14:39 2022] DELETE /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Aug 07 01:14:39.535498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f680ab2b-a361-4458-a22e-95304834feed tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:39.536263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f680ab2b-a361-4458-a22e-95304834feed tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:39.542969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f680ab2b-a361-4458-a22e-95304834feed tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:39.543197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f680ab2b-a361-4458-a22e-95304834feed tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:39.543631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f680ab2b-a361-4458-a22e-95304834feed tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:39.544399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f680ab2b-a361-4458-a22e-95304834feed tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:39.544858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f680ab2b-a361-4458-a22e-95304834feed tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:39.545402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f680ab2b-a361-4458-a22e-95304834feed tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:39.613536 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-693e41ba-dadf-46ac-9279-db9f8ccea300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:39.630364 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-693e41ba-dadf-46ac-9279-db9f8ccea300 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.246692 Aug 07 01:14:39.631415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 279/580] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:39 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:39.727140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f680ab2b-a361-4458-a22e-95304834feed tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:39.739741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f680ab2b-a361-4458-a22e-95304834feed tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:39.741523 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f680ab2b-a361-4458-a22e-95304834feed tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:39.742862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f680ab2b-a361-4458-a22e-95304834feed tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.212603 Aug 07 01:14:39.743456 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 302/581] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:14:39 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:14:40.409778 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1605d915-304c-46f4-a7bc-b087a30d8629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:40.410913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1605d915-304c-46f4-a7bc-b087a30d8629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:40.422532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1605d915-304c-46f4-a7bc-b087a30d8629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:40.423716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1605d915-304c-46f4-a7bc-b087a30d8629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:40.424268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1605d915-304c-46f4-a7bc-b087a30d8629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:40.425139 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1605d915-304c-46f4-a7bc-b087a30d8629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:40.425697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1605d915-304c-46f4-a7bc-b087a30d8629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:40.426128 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1605d915-304c-46f4-a7bc-b087a30d8629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:40.635079 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1605d915-304c-46f4-a7bc-b087a30d8629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:40.643558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1605d915-304c-46f4-a7bc-b087a30d8629 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.241560 Aug 07 01:14:40.644219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 280/582] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:40 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:40.664350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4cca13ff-8082-4bc2-9558-89acc3e7af7a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:40.665011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4cca13ff-8082-4bc2-9558-89acc3e7af7a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:40.673406 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4cca13ff-8082-4bc2-9558-89acc3e7af7a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:40.673926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4cca13ff-8082-4bc2-9558-89acc3e7af7a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:40.674370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4cca13ff-8082-4bc2-9558-89acc3e7af7a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:40.675123 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4cca13ff-8082-4bc2-9558-89acc3e7af7a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:40.675684 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4cca13ff-8082-4bc2-9558-89acc3e7af7a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:40.676454 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4cca13ff-8082-4bc2-9558-89acc3e7af7a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:40.778145 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b880b4f6-e8df-475e-9643-8a3f94adceca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:40.780040 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b880b4f6-e8df-475e-9643-8a3f94adceca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:40.790178 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b880b4f6-e8df-475e-9643-8a3f94adceca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:40.791199 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b880b4f6-e8df-475e-9643-8a3f94adceca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:40.791793 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b880b4f6-e8df-475e-9643-8a3f94adceca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:40.793080 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b880b4f6-e8df-475e-9643-8a3f94adceca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:40.793891 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b880b4f6-e8df-475e-9643-8a3f94adceca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:40.794650 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b880b4f6-e8df-475e-9643-8a3f94adceca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:40.865600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4cca13ff-8082-4bc2-9558-89acc3e7af7a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:40.878704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4cca13ff-8082-4bc2-9558-89acc3e7af7a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.218187 Aug 07 01:14:40.879833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 303/583] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:40 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:40.888791 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-996fff29-6ea0-47c6-b5a5-bbf0963a0225 req-23d1ea43-0b54-464c-b7ea-f54b77e94ee3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9968bbaa-9908-4063-bc7f-e6e805385fa2", "name": "network-vif-plugged", "status": "completed", "tag": "6681d8c9-7955-47db-a880-4ab2cb9ac6d1"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:40.907789 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-996fff29-6ea0-47c6-b5a5-bbf0963a0225 req-23d1ea43-0b54-464c-b7ea-f54b77e94ee3 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:40.908678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-996fff29-6ea0-47c6-b5a5-bbf0963a0225 req-23d1ea43-0b54-464c-b7ea-f54b77e94ee3 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:40.909489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-996fff29-6ea0-47c6-b5a5-bbf0963a0225 req-23d1ea43-0b54-464c-b7ea-f54b77e94ee3 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:40.932159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-996fff29-6ea0-47c6-b5a5-bbf0963a0225 req-23d1ea43-0b54-464c-b7ea-f54b77e94ee3 service nova] Creating event network-vif-plugged:6681d8c9-7955-47db-a880-4ab2cb9ac6d1 for instance 9968bbaa-9908-4063-bc7f-e6e805385fa2 on hv-cinder-maste Aug 07 01:14:40.941185 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-996fff29-6ea0-47c6-b5a5-bbf0963a0225 req-23d1ea43-0b54-464c-b7ea-f54b77e94ee3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059571 Aug 07 01:14:40.941810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 304/584] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:14:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:14:40.950310 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9e836e82-6627-480f-b441-06c0ff71d316 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:40.950990 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9e836e82-6627-480f-b441-06c0ff71d316 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:40.965273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e836e82-6627-480f-b441-06c0ff71d316 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:40.966141 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e836e82-6627-480f-b441-06c0ff71d316 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:40.966926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e836e82-6627-480f-b441-06c0ff71d316 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:40.968182 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e836e82-6627-480f-b441-06c0ff71d316 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:40.968980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e836e82-6627-480f-b441-06c0ff71d316 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:40.969627 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e836e82-6627-480f-b441-06c0ff71d316 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:40.971504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b880b4f6-e8df-475e-9643-8a3f94adceca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:40.986884 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b880b4f6-e8df-475e-9643-8a3f94adceca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:40.990230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b880b4f6-e8df-475e-9643-8a3f94adceca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:40.992725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b880b4f6-e8df-475e-9643-8a3f94adceca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.221275 Aug 07 01:14:40.993855 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 281/585] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:14:40 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:14:41.076528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.objects.instance [None req-9e836e82-6627-480f-b441-06c0ff71d316 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lazy-loading 'services' on Instance uuid 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc {{(pid=117983) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:14:41.175152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9e836e82-6627-480f-b441-06c0ff71d316 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "DELETE /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc/os-volume_attachments/5ed02bd8-a8d9-42b9-9d85-f4c9fe9f60ff" status: 202 len: 0 microversion: 2.1 time: 0.231780 Aug 07 01:14:41.175986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 305/586] 10.222.0.26 () {58 vars in 1416 bytes} [Sun Aug 7 01:14:40 2022] DELETE /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc/os-volume_attachments/5ed02bd8-a8d9-42b9-9d85-f4c9fe9f60ff => generated 0 bytes in 234 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:14:41.667137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0dc5b4f3-e204-49c6-863f-3bc1927005d5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:41.668251 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0dc5b4f3-e204-49c6-863f-3bc1927005d5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:41.676927 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0dc5b4f3-e204-49c6-863f-3bc1927005d5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:41.677763 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0dc5b4f3-e204-49c6-863f-3bc1927005d5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:41.678463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0dc5b4f3-e204-49c6-863f-3bc1927005d5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:41.679493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0dc5b4f3-e204-49c6-863f-3bc1927005d5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:41.680045 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0dc5b4f3-e204-49c6-863f-3bc1927005d5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:41.680615 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0dc5b4f3-e204-49c6-863f-3bc1927005d5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:41.885754 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0dc5b4f3-e204-49c6-863f-3bc1927005d5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:41.895885 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0dc5b4f3-e204-49c6-863f-3bc1927005d5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.233577 Aug 07 01:14:41.896539 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 282/587] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:41 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:41.910575 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-413cdef0-e72a-4c26-a2ac-3979ade8e93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:41.913645 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-413cdef0-e72a-4c26-a2ac-3979ade8e93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:41.924377 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-413cdef0-e72a-4c26-a2ac-3979ade8e93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:41.925309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-413cdef0-e72a-4c26-a2ac-3979ade8e93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:41.926099 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-413cdef0-e72a-4c26-a2ac-3979ade8e93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:41.927392 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-413cdef0-e72a-4c26-a2ac-3979ade8e93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:41.930274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-413cdef0-e72a-4c26-a2ac-3979ade8e93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:41.930274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-413cdef0-e72a-4c26-a2ac-3979ade8e93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:42.040415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b097de36-206c-4193-9c87-5775d4d8e6ec tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:42.041481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b097de36-206c-4193-9c87-5775d4d8e6ec tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:42.054739 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b097de36-206c-4193-9c87-5775d4d8e6ec tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:42.055586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b097de36-206c-4193-9c87-5775d4d8e6ec tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:42.056547 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b097de36-206c-4193-9c87-5775d4d8e6ec tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:42.059026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b097de36-206c-4193-9c87-5775d4d8e6ec tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:42.059413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b097de36-206c-4193-9c87-5775d4d8e6ec tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:42.060415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b097de36-206c-4193-9c87-5775d4d8e6ec tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:42.163191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-413cdef0-e72a-4c26-a2ac-3979ade8e93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:42.174614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-413cdef0-e72a-4c26-a2ac-3979ade8e93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.273055 Aug 07 01:14:42.175403 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 306/588] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:41 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:42.294146 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b097de36-206c-4193-9c87-5775d4d8e6ec tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:42.306148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b097de36-206c-4193-9c87-5775d4d8e6ec tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:42.308323 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b097de36-206c-4193-9c87-5775d4d8e6ec tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:42.309595 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b097de36-206c-4193-9c87-5775d4d8e6ec tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.278817 Aug 07 01:14:42.310169 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 283/589] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:14:42 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:14:42.925460 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ac5773ac-7d40-4df3-8c71-ec17364b6fc8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:42.926985 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ac5773ac-7d40-4df3-8c71-ec17364b6fc8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:42.942631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac5773ac-7d40-4df3-8c71-ec17364b6fc8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:42.944560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac5773ac-7d40-4df3-8c71-ec17364b6fc8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:42.945924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac5773ac-7d40-4df3-8c71-ec17364b6fc8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:42.947521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac5773ac-7d40-4df3-8c71-ec17364b6fc8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:42.948885 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac5773ac-7d40-4df3-8c71-ec17364b6fc8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:42.949671 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac5773ac-7d40-4df3-8c71-ec17364b6fc8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:43.213632 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bf24d53d-3fa6-46fa-99c3-032ea9ef6a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:43.214227 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bf24d53d-3fa6-46fa-99c3-032ea9ef6a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:43.223182 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=3 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:43.231358 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bf24d53d-3fa6-46fa-99c3-032ea9ef6a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:43.232508 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bf24d53d-3fa6-46fa-99c3-032ea9ef6a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:43.233369 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bf24d53d-3fa6-46fa-99c3-032ea9ef6a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:43.237702 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bf24d53d-3fa6-46fa-99c3-032ea9ef6a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:43.238347 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bf24d53d-3fa6-46fa-99c3-032ea9ef6a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:43.239002 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bf24d53d-3fa6-46fa-99c3-032ea9ef6a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:43.240223 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ac5773ac-7d40-4df3-8c71-ec17364b6fc8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:43.289059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ac5773ac-7d40-4df3-8c71-ec17364b6fc8 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1367 microversion: 2.1 time: 0.370467 Aug 07 01:14:43.293122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 307/590] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:42 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1367 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:43.341665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-af22ccf3-e06f-4a7f-806c-edea550900c9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:43.343260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-af22ccf3-e06f-4a7f-806c-edea550900c9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:43.357905 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af22ccf3-e06f-4a7f-806c-edea550900c9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:43.358866 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af22ccf3-e06f-4a7f-806c-edea550900c9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:43.360323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af22ccf3-e06f-4a7f-806c-edea550900c9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:43.361670 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af22ccf3-e06f-4a7f-806c-edea550900c9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:43.363265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af22ccf3-e06f-4a7f-806c-edea550900c9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:43.364485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af22ccf3-e06f-4a7f-806c-edea550900c9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:43.623118 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bf24d53d-3fa6-46fa-99c3-032ea9ef6a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:43.631549 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-af22ccf3-e06f-4a7f-806c-edea550900c9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:43.639060 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bf24d53d-3fa6-46fa-99c3-032ea9ef6a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.434319 Aug 07 01:14:43.640128 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 284/591] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:43 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:43.662737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-af22ccf3-e06f-4a7f-806c-edea550900c9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:43.664484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-af22ccf3-e06f-4a7f-806c-edea550900c9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:43.665734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-af22ccf3-e06f-4a7f-806c-edea550900c9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1723 microversion: 2.20 time: 0.332287 Aug 07 01:14:43.666528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 308/592] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:14:43 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1723 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:14:44.333551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-89e9df30-d2a9-447b-9661-bff5e68df488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:44.334570 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-89e9df30-d2a9-447b-9661-bff5e68df488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:44.343258 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89e9df30-d2a9-447b-9661-bff5e68df488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:44.343801 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89e9df30-d2a9-447b-9661-bff5e68df488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:44.344278 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89e9df30-d2a9-447b-9661-bff5e68df488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:44.345077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89e9df30-d2a9-447b-9661-bff5e68df488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:44.345731 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89e9df30-d2a9-447b-9661-bff5e68df488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:44.346163 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89e9df30-d2a9-447b-9661-bff5e68df488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:44.599753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7d1b9ea9-b6ee-494a-8d05-f6ecd8f6a411 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:44.600640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7d1b9ea9-b6ee-494a-8d05-f6ecd8f6a411 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:44.606685 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:44.615989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d1b9ea9-b6ee-494a-8d05-f6ecd8f6a411 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:44.617360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d1b9ea9-b6ee-494a-8d05-f6ecd8f6a411 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:44.618444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d1b9ea9-b6ee-494a-8d05-f6ecd8f6a411 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:44.620019 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d1b9ea9-b6ee-494a-8d05-f6ecd8f6a411 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:44.620895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d1b9ea9-b6ee-494a-8d05-f6ecd8f6a411 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:44.621643 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d1b9ea9-b6ee-494a-8d05-f6ecd8f6a411 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:44.673934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7d1b9ea9-b6ee-494a-8d05-f6ecd8f6a411 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Going to try to terminate instance {{(pid=117983) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:14:44.699227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:44.701155 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-89e9df30-d2a9-447b-9661-bff5e68df488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:44.714418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-89e9df30-d2a9-447b-9661-bff5e68df488 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1558 microversion: 2.1 time: 0.385360 Aug 07 01:14:44.715534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 285/593] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:44 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1558 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:44.732527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0a25750a-23ff-4713-9fc7-da6dae4313fd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:44.733523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0a25750a-23ff-4713-9fc7-da6dae4313fd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:44.740733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0a25750a-23ff-4713-9fc7-da6dae4313fd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:44.741591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0a25750a-23ff-4713-9fc7-da6dae4313fd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:44.742051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0a25750a-23ff-4713-9fc7-da6dae4313fd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:44.742902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0a25750a-23ff-4713-9fc7-da6dae4313fd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:44.743373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0a25750a-23ff-4713-9fc7-da6dae4313fd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:44.743806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0a25750a-23ff-4713-9fc7-da6dae4313fd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:44.829960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7d1b9ea9-b6ee-494a-8d05-f6ecd8f6a411 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "DELETE /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 204 len: 0 microversion: 2.1 time: 0.236062 Aug 07 01:14:44.832194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 309/594] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:14:44 2022] DELETE /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 0 bytes in 241 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 07 01:14:44.843095 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e01abee5-adb0-45af-ba50-688512131768 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:44.844328 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e01abee5-adb0-45af-ba50-688512131768 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:44.855759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e01abee5-adb0-45af-ba50-688512131768 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:44.856731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e01abee5-adb0-45af-ba50-688512131768 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:44.857771 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e01abee5-adb0-45af-ba50-688512131768 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:44.858984 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e01abee5-adb0-45af-ba50-688512131768 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:44.864440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e01abee5-adb0-45af-ba50-688512131768 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:44.865097 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e01abee5-adb0-45af-ba50-688512131768 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:44.972698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e01abee5-adb0-45af-ba50-688512131768 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:44.990681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e01abee5-adb0-45af-ba50-688512131768 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:44.994704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e01abee5-adb0-45af-ba50-688512131768 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:44.997857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e01abee5-adb0-45af-ba50-688512131768 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1655 microversion: 2.20 time: 0.162745 Aug 07 01:14:44.999036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 310/595] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:14:44 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1655 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:14:45.009232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e906e792-f044-430d-8705-e9ad0d1f3e10 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:45.010271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e906e792-f044-430d-8705-e9ad0d1f3e10 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:45.026293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e906e792-f044-430d-8705-e9ad0d1f3e10 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:45.027358 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e906e792-f044-430d-8705-e9ad0d1f3e10 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:45.028461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e906e792-f044-430d-8705-e9ad0d1f3e10 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:45.031007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e906e792-f044-430d-8705-e9ad0d1f3e10 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:45.031854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e906e792-f044-430d-8705-e9ad0d1f3e10 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:45.033152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e906e792-f044-430d-8705-e9ad0d1f3e10 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:45.036742 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0a25750a-23ff-4713-9fc7-da6dae4313fd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:45.053958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0a25750a-23ff-4713-9fc7-da6dae4313fd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1266 microversion: 2.1 time: 0.332189 Aug 07 01:14:45.055850 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 286/596] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:44 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1266 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:45.065720 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-29b9c9b7-5ea8-4d34-8b2e-59f3f04bc02e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:45.067234 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-29b9c9b7-5ea8-4d34-8b2e-59f3f04bc02e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:45.080610 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-29b9c9b7-5ea8-4d34-8b2e-59f3f04bc02e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:45.081487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-29b9c9b7-5ea8-4d34-8b2e-59f3f04bc02e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:45.082282 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-29b9c9b7-5ea8-4d34-8b2e-59f3f04bc02e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:45.083601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-29b9c9b7-5ea8-4d34-8b2e-59f3f04bc02e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:45.084753 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-29b9c9b7-5ea8-4d34-8b2e-59f3f04bc02e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:45.085596 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-29b9c9b7-5ea8-4d34-8b2e-59f3f04bc02e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:45.155653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e906e792-f044-430d-8705-e9ad0d1f3e10 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.154932 Aug 07 01:14:45.160854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 311/597] 10.222.0.26 () {58 vars in 1275 bytes} [Sun Aug 7 01:14:44 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222/os-interface => generated 285 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:14:45.163820 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-19d55289-ae37-4db4-aed4-9fab57b8019b req-ce0378ac-4e7f-4550-ae70-4c33b52491cf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7f60fb22-d17a-4b7c-9b7d-95543be1907e", "name": "network-vif-deleted", "tag": "9ab59ce3-ed0d-4a8e-b76b-adeff559725a"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:45.176352 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-19d55289-ae37-4db4-aed4-9fab57b8019b req-ce0378ac-4e7f-4550-ae70-4c33b52491cf service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:45.177285 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-19d55289-ae37-4db4-aed4-9fab57b8019b req-ce0378ac-4e7f-4550-ae70-4c33b52491cf service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:45.177828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-19d55289-ae37-4db4-aed4-9fab57b8019b req-ce0378ac-4e7f-4550-ae70-4c33b52491cf service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:45.195715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-19d55289-ae37-4db4-aed4-9fab57b8019b req-ce0378ac-4e7f-4550-ae70-4c33b52491cf service nova] Creating event network-vif-deleted:9ab59ce3-ed0d-4a8e-b76b-adeff559725a for instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e on hv-cinder-maste Aug 07 01:14:45.201758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-19d55289-ae37-4db4-aed4-9fab57b8019b req-ce0378ac-4e7f-4550-ae70-4c33b52491cf service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043444 Aug 07 01:14:45.203395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 312/598] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:14:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:14:45.325074 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-29b9c9b7-5ea8-4d34-8b2e-59f3f04bc02e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:45.338468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-29b9c9b7-5ea8-4d34-8b2e-59f3f04bc02e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1513 microversion: 2.1 time: 0.280197 Aug 07 01:14:45.339268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 287/599] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:45 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1513 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:46.043261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a0aecf9e-8931-49b5-a173-6835f42e099d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:46.044436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a0aecf9e-8931-49b5-a173-6835f42e099d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:46.054575 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0aecf9e-8931-49b5-a173-6835f42e099d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:46.055901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0aecf9e-8931-49b5-a173-6835f42e099d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:46.056743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0aecf9e-8931-49b5-a173-6835f42e099d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:46.057877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0aecf9e-8931-49b5-a173-6835f42e099d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:46.058604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0aecf9e-8931-49b5-a173-6835f42e099d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:46.059661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0aecf9e-8931-49b5-a173-6835f42e099d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:46.078574 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fcb26806-590e-428a-8a98-72d7b0b71b28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:46.080543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fcb26806-590e-428a-8a98-72d7b0b71b28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:46.088294 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fcb26806-590e-428a-8a98-72d7b0b71b28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:46.089153 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fcb26806-590e-428a-8a98-72d7b0b71b28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:46.089772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fcb26806-590e-428a-8a98-72d7b0b71b28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:46.090815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fcb26806-590e-428a-8a98-72d7b0b71b28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:46.091457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fcb26806-590e-428a-8a98-72d7b0b71b28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:46.092073 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fcb26806-590e-428a-8a98-72d7b0b71b28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:46.206646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a0aecf9e-8931-49b5-a173-6835f42e099d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:46.217806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a0aecf9e-8931-49b5-a173-6835f42e099d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:46.219445 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a0aecf9e-8931-49b5-a173-6835f42e099d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:46.220996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a0aecf9e-8931-49b5-a173-6835f42e099d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1364 microversion: 2.20 time: 0.189210 Aug 07 01:14:46.221946 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 313/600] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:14:46 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1364 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:14:46.283708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fcb26806-590e-428a-8a98-72d7b0b71b28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:46.299643 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fcb26806-590e-428a-8a98-72d7b0b71b28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1452 microversion: 2.1 time: 0.226663 Aug 07 01:14:46.300351 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 288/601] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:46 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1452 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:46.335744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-54068adf-5f24-4d34-b25a-3710b7e9839a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:46.336729 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-54068adf-5f24-4d34-b25a-3710b7e9839a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:46.349513 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54068adf-5f24-4d34-b25a-3710b7e9839a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:46.350390 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54068adf-5f24-4d34-b25a-3710b7e9839a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:46.358785 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54068adf-5f24-4d34-b25a-3710b7e9839a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:46.358785 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54068adf-5f24-4d34-b25a-3710b7e9839a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:46.358785 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54068adf-5f24-4d34-b25a-3710b7e9839a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:46.358785 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54068adf-5f24-4d34-b25a-3710b7e9839a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:46.363191 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-81ddf2f0-3dbf-4765-a60d-b48f984ab123 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:46.364332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-81ddf2f0-3dbf-4765-a60d-b48f984ab123 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:46.378590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-81ddf2f0-3dbf-4765-a60d-b48f984ab123 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:46.379328 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-81ddf2f0-3dbf-4765-a60d-b48f984ab123 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:46.379788 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-81ddf2f0-3dbf-4765-a60d-b48f984ab123 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:46.380635 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-81ddf2f0-3dbf-4765-a60d-b48f984ab123 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:46.381087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-81ddf2f0-3dbf-4765-a60d-b48f984ab123 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:46.381993 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-81ddf2f0-3dbf-4765-a60d-b48f984ab123 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:46.633452 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-81ddf2f0-3dbf-4765-a60d-b48f984ab123 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:46.643090 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-81ddf2f0-3dbf-4765-a60d-b48f984ab123 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1513 microversion: 2.1 time: 0.285199 Aug 07 01:14:46.643689 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 289/602] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:46 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1513 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:14:46.931762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-54068adf-5f24-4d34-b25a-3710b7e9839a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:46.946991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-54068adf-5f24-4d34-b25a-3710b7e9839a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1452 microversion: 2.1 time: 0.617636 Aug 07 01:14:46.948563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 314/603] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:46 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1452 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:47.133768 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-47a4237f-b948-4650-ae9a-0e13def9cc7f req-296c60b8-65a7-4976-94d6-2b3997263330 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e86efc1a-b689-454a-8683-526a9c18e222", "tag": "f5f83050-c635-4216-9f73-3f7cd60024ff"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:47.147174 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-47a4237f-b948-4650-ae9a-0e13def9cc7f req-296c60b8-65a7-4976-94d6-2b3997263330 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:47.147664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-47a4237f-b948-4650-ae9a-0e13def9cc7f req-296c60b8-65a7-4976-94d6-2b3997263330 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:47.148142 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-47a4237f-b948-4650-ae9a-0e13def9cc7f req-296c60b8-65a7-4976-94d6-2b3997263330 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:47.165413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-47a4237f-b948-4650-ae9a-0e13def9cc7f req-296c60b8-65a7-4976-94d6-2b3997263330 service nova] Creating event network-changed:f5f83050-c635-4216-9f73-3f7cd60024ff for instance e86efc1a-b689-454a-8683-526a9c18e222 on hv-cinder-maste Aug 07 01:14:47.171764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-47a4237f-b948-4650-ae9a-0e13def9cc7f req-296c60b8-65a7-4976-94d6-2b3997263330 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042098 Aug 07 01:14:47.173877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 290/604] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:14:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:14:47.263866 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:47.265359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:47.276011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:47.277431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:47.278384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:47.280071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:47.281162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:47.282751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:47.363401 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:47.364901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:47.367040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:47.368525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.objects.instance [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lazy-loading 'fault' on Instance uuid 7f60fb22-d17a-4b7c-9b7d-95543be1907e {{(pid=117983) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:14:47.481493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:47.491656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:47.493440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:47.494658 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-83c9e5ed-092e-4644-a3f2-9b62917ec5ba tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 200 len: 1369 microversion: 2.20 time: 0.236175 Aug 07 01:14:47.495523 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 315/605] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:14:47 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 1369 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:14:47.672495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-02c612cd-35e4-4919-9135-079ed7c78a03 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:47.673807 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-02c612cd-35e4-4919-9135-079ed7c78a03 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:47.682413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02c612cd-35e4-4919-9135-079ed7c78a03 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:47.682413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02c612cd-35e4-4919-9135-079ed7c78a03 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:47.682413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02c612cd-35e4-4919-9135-079ed7c78a03 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:47.683660 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02c612cd-35e4-4919-9135-079ed7c78a03 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:47.684651 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02c612cd-35e4-4919-9135-079ed7c78a03 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:47.685422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02c612cd-35e4-4919-9135-079ed7c78a03 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:48.098133 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-02c612cd-35e4-4919-9135-079ed7c78a03 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:48.117711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-02c612cd-35e4-4919-9135-079ed7c78a03 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 200 len: 1513 microversion: 2.1 time: 0.448863 Aug 07 01:14:48.120656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 291/606] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:47 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 1513 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:14:48.538312 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a7ffb1f5-902d-4e36-a254-7551b5724507 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:48.540110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a7ffb1f5-902d-4e36-a254-7551b5724507 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:48.553516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7ffb1f5-902d-4e36-a254-7551b5724507 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:48.554761 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7ffb1f5-902d-4e36-a254-7551b5724507 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:48.556011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7ffb1f5-902d-4e36-a254-7551b5724507 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:48.558181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7ffb1f5-902d-4e36-a254-7551b5724507 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:48.559258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7ffb1f5-902d-4e36-a254-7551b5724507 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:48.559923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7ffb1f5-902d-4e36-a254-7551b5724507 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:48.581762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-a7ffb1f5-902d-4e36-a254-7551b5724507 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] HTTP exception thrown: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a7ffb1f5-902d-4e36-a254-7551b5724507 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Request method failure captured: Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e HTTP/1.1 Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xDuZbhbLDbrOLrZVY9vlm_HtWVxIubBKHAStAP1FnjiA_U5zjLpQJv9uy1CLGRNLdulE-dSDkxwztkKA7ofKoolemIQ0OeMpjkkOauZx4Jr-U23jKoErXsQ2oRdJqqP6y71FTJ9znyT4ExNnLWW_EBEi4UJWyaqL-JEhBPxpiLjkk0 Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Openstack-Nova-Api-Version: 2.20 Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 1b1d33b8c71e41e6bf23d4ad8590d2c2 Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:14:48.584161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:14:48.587396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:14:48.587396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:14:48.587396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 1b1d33b8c71e41e6bf23d4ad8590d2c2 Aug 07 01:14:48.587396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:14:48.587396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-AttachVolumeShelveTestJSON-734115863-project Aug 07 01:14:48.587396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:14:48.587396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:14:48.587396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 5fafe42957914b929f9ffee53ab6e7ac Aug 07 01:14:48.587396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-AttachVolumeShelveTestJSON-734115863-project Aug 07 01:14:48.587396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:14:48.587396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:14:48.587396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '7f60fb22-d17a-4b7c-9b7d-95543be1907e'} Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:14:48.589433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:14:48.592848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:14:48.596383 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a7ffb1f5-902d-4e36-a254-7551b5724507 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Returning 404 to user: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:14:48.596383 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a7ffb1f5-902d-4e36-a254-7551b5724507 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 404 len: 111 microversion: 2.20 time: 0.054552 Aug 07 01:14:48.602399 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a276355e-6f16-4547-830a-04881d214402 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '.version_select at 0x7f1fad34eaf0>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:48.605707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 316/607] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:14:48 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 111 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 07 01:14:48.628452 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a276355e-6f16-4547-830a-04881d214402 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-7392549" status: 204 len: 0 microversion: 2.20 time: 0.032299 Aug 07 01:14:48.629684 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 292/608] 10.222.0.26 () {60 vars in 1253 bytes} [Sun Aug 7 01:14:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-7392549 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 07 01:14:49.151092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-54c84791-1a87-4d0d-907d-1a7964a27d94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:49.152165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-54c84791-1a87-4d0d-907d-1a7964a27d94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:49.189294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54c84791-1a87-4d0d-907d-1a7964a27d94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:49.204973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-0fc81a3e-e43b-454d-ae54-90984510cc2b req-bfe85ef0-8af1-4399-8acb-c47460d98824 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc", "name": "network-vif-deleted", "tag": "3ba58955-f906-4b1b-8224-8a7408f3a52c"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:49.210463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54c84791-1a87-4d0d-907d-1a7964a27d94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:49.211734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54c84791-1a87-4d0d-907d-1a7964a27d94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.021s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:49.213227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54c84791-1a87-4d0d-907d-1a7964a27d94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:49.217527 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54c84791-1a87-4d0d-907d-1a7964a27d94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:49.218516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54c84791-1a87-4d0d-907d-1a7964a27d94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:49.242949 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-0fc81a3e-e43b-454d-ae54-90984510cc2b req-bfe85ef0-8af1-4399-8acb-c47460d98824 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:49.243429 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-54c84791-1a87-4d0d-907d-1a7964a27d94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] HTTP exception thrown: Instance 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc could not be found. Aug 07 01:14:49.244786 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-0fc81a3e-e43b-454d-ae54-90984510cc2b req-bfe85ef0-8af1-4399-8acb-c47460d98824 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-54c84791-1a87-4d0d-907d-1a7964a27d94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Request method failure captured: Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc HTTP/1.1 Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xGwdVHZVWnHIzGDSpTVNiz-q980RAQtQDxjuRjGuqs_HJoXOXoBhZ0fF1bbTTjk8m9Pvp2IcKkTppafVNQBMvnpaEQe9WwCTTgrFUCyEMZcpdieiPEfKYhmeOU5srzGG4O4YeIgb7xO3BdRG6OZeDumpyuz3KQXJRzGLfuf1xxtaZM Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: f7b08cc6d4a34961bdf464b58946a02e Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:14:49.245434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:14:49.247006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:14:49.247006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:14:49.247006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: f7b08cc6d4a34961bdf464b58946a02e Aug 07 01:14:49.247006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:14:49.247006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-VolumesSnapshotTestJSON-2086260047-project Aug 07 01:14:49.247006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:14:49.247006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:14:49.247006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 6dd0ed2144164190b380ed58b3dd7cd4 Aug 07 01:14:49.247006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-VolumesSnapshotTestJSON-2086260047-project Aug 07 01:14:49.247006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:14:49.247006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc could not be found. Aug 07 01:14:49.247006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc'} Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc could not be found. Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:14:49.247963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc could not be found. Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc could not be found. Aug 07 01:14:49.249166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:14:49.250225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-0fc81a3e-e43b-454d-ae54-90984510cc2b req-bfe85ef0-8af1-4399-8acb-c47460d98824 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:49.252154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-54c84791-1a87-4d0d-907d-1a7964a27d94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Returning 404 to user: Instance 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:14:49.253489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-54c84791-1a87-4d0d-907d-1a7964a27d94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc" status: 404 len: 111 microversion: 2.1 time: 0.112384 Aug 07 01:14:49.254681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 317/609] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:14:49 2022] GET /compute/v2.1/servers/35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc => generated 111 bytes in 117 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:14:49.257319 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:14:49.292527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-0fc81a3e-e43b-454d-ae54-90984510cc2b req-bfe85ef0-8af1-4399-8acb-c47460d98824 service nova] Creating event network-vif-deleted:3ba58955-f906-4b1b-8224-8a7408f3a52c for instance 35bc5ab3-2492-4ba1-a5b1-e4bd337c8fdc on hv-cinder-maste Aug 07 01:14:49.308122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-0fc81a3e-e43b-454d-ae54-90984510cc2b req-bfe85ef0-8af1-4399-8acb-c47460d98824 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.114281 Aug 07 01:14:49.311915 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 293/610] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:14:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:14:49.902613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dc61c147-8c34-4550-88bb-649e2019036d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Action: 'create', calling method: .version_select at 0x7f1fad721a60>, body: {"keypair": {"name": "tempest-keypair-672649657", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJLPCydfS9GFs3G1K44LvgNRVBPSVurzYwA8htnhmlM3eZjgVj9W06NtXo/kPilt3CBQfLCc4UAZPhm+b8MXANrSHvrgiwwXr84loa1TdmdNvhCblAuyoRU/cc1n56WeSA=="}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:49.937233 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-dc61c147-8c34-4550-88bb-649e2019036d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Getting quotas for project 1b1d33b8c71e41e6bf23d4ad8590d2c2. Resources: {'key_pairs'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:14:49.945241 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-dc61c147-8c34-4550-88bb-649e2019036d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Getting quotas for user 5fafe42957914b929f9ffee53ab6e7ac and project 1b1d33b8c71e41e6bf23d4ad8590d2c2. Resources: {'key_pairs'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:14:49.989016 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dc61c147-8c34-4550-88bb-649e2019036d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.097829 Aug 07 01:14:50.000579 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 318/611] 10.222.0.26 () {62 vars in 1200 bytes} [Sun Aug 7 01:14:49 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 101 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Aug 07 01:14:50.288181 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-f348ad37-ebd8-453a-bccd-c833788c10ba req-25d6256a-5d10-4c2c-ae3f-d28b18f3e62e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9968bbaa-9908-4063-bc7f-e6e805385fa2", "tag": "6681d8c9-7955-47db-a880-4ab2cb9ac6d1"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:50.302465 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-f348ad37-ebd8-453a-bccd-c833788c10ba req-25d6256a-5d10-4c2c-ae3f-d28b18f3e62e service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:50.303224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-f348ad37-ebd8-453a-bccd-c833788c10ba req-25d6256a-5d10-4c2c-ae3f-d28b18f3e62e service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:50.304168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-f348ad37-ebd8-453a-bccd-c833788c10ba req-25d6256a-5d10-4c2c-ae3f-d28b18f3e62e service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:50.318287 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-f348ad37-ebd8-453a-bccd-c833788c10ba req-25d6256a-5d10-4c2c-ae3f-d28b18f3e62e service nova] Creating event network-changed:6681d8c9-7955-47db-a880-4ab2cb9ac6d1 for instance 9968bbaa-9908-4063-bc7f-e6e805385fa2 on hv-cinder-maste Aug 07 01:14:50.324771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-f348ad37-ebd8-453a-bccd-c833788c10ba req-25d6256a-5d10-4c2c-ae3f-d28b18f3e62e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041426 Aug 07 01:14:50.326184 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 294/612] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:14:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:14:52.107473 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1121970568", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "96bfca6e-7131-4db0-b142-73ffbffe52dd"}], "security_groups": [{"name": "tempest-securitygroup--1029569415"}], "key_name": "tempest-keypair-672649657", "user_data": "}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:52.237215 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.network.neutron [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] validate_networks() for [('96bfca6e-7131-4db0-b142-73ffbffe52dd', None, None, None, None, None)] {{(pid=117983) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:14:52.783262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Flavor limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:14:52.784068 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Image limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:14:52.785025 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Flavor pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:14:52.785979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Image pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:14:52.786774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:14:52.804350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:52.805222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:52.806039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:52.807484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:52.808379 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:52.809187 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:52.851032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Getting quotas for project 1b1d33b8c71e41e6bf23d4ad8590d2c2. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:14:52.862420 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Getting quotas for user 5fafe42957914b929f9ffee53ab6e7ac and project 1b1d33b8c71e41e6bf23d4ad8590d2c2. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:14:52.887368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Going to run 1 instances... {{(pid=117983) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:14:52.894602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117983) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:14:52.977748 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-02e6f43b-7c3b-4cca-b9fd-18d26655093a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.874660 Aug 07 01:14:52.978782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 319/613] 10.222.0.26 () {62 vars in 1188 bytes} [Sun Aug 7 01:14:52 2022] POST /compute/v2.1/servers => generated 402 bytes in 876 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Aug 07 01:14:52.999048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-40eb45fe-aa91-4332-b933-769e9ed6dbc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:53.000295 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-40eb45fe-aa91-4332-b933-769e9ed6dbc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:53.148864 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-40eb45fe-aa91-4332-b933-769e9ed6dbc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:53.163787 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-40eb45fe-aa91-4332-b933-769e9ed6dbc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:53.166653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-40eb45fe-aa91-4332-b933-769e9ed6dbc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:53.168591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-40eb45fe-aa91-4332-b933-769e9ed6dbc4 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1281 microversion: 2.20 time: 0.177594 Aug 07 01:14:53.169997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 295/614] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:14:52 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1281 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:14:54.201479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c650bc27-f33a-48be-ae3b-fda6f35b4760 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:54.202623 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c650bc27-f33a-48be-ae3b-fda6f35b4760 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:54.210423 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c650bc27-f33a-48be-ae3b-fda6f35b4760 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:54.210922 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c650bc27-f33a-48be-ae3b-fda6f35b4760 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:54.211370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c650bc27-f33a-48be-ae3b-fda6f35b4760 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:54.212103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c650bc27-f33a-48be-ae3b-fda6f35b4760 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:54.212608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c650bc27-f33a-48be-ae3b-fda6f35b4760 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:54.213052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c650bc27-f33a-48be-ae3b-fda6f35b4760 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:54.298103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c650bc27-f33a-48be-ae3b-fda6f35b4760 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:54.307625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c650bc27-f33a-48be-ae3b-fda6f35b4760 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:54.309222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c650bc27-f33a-48be-ae3b-fda6f35b4760 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:54.310391 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c650bc27-f33a-48be-ae3b-fda6f35b4760 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1333 microversion: 2.20 time: 0.114414 Aug 07 01:14:54.311291 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 320/615] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:14:54 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1333 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:14:55.345401 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4bbd5072-917b-4e99-b4e9-366d293777fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:55.347343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4bbd5072-917b-4e99-b4e9-366d293777fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:55.362776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4bbd5072-917b-4e99-b4e9-366d293777fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:55.363969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4bbd5072-917b-4e99-b4e9-366d293777fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:55.364460 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4bbd5072-917b-4e99-b4e9-366d293777fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:55.365494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4bbd5072-917b-4e99-b4e9-366d293777fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:55.366087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4bbd5072-917b-4e99-b4e9-366d293777fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:55.366767 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4bbd5072-917b-4e99-b4e9-366d293777fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:55.451422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4bbd5072-917b-4e99-b4e9-366d293777fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:55.462367 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4bbd5072-917b-4e99-b4e9-366d293777fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:55.463818 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4bbd5072-917b-4e99-b4e9-366d293777fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:55.465230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4bbd5072-917b-4e99-b4e9-366d293777fc tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1339 microversion: 2.20 time: 0.128112 Aug 07 01:14:55.466169 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 296/616] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:14:55 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1339 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:14:56.496653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-19b00657-f599-4593-a67c-4d138e0b0026 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:56.497306 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-19b00657-f599-4593-a67c-4d138e0b0026 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:56.504833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19b00657-f599-4593-a67c-4d138e0b0026 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:56.505322 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19b00657-f599-4593-a67c-4d138e0b0026 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:56.505760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19b00657-f599-4593-a67c-4d138e0b0026 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:56.506472 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19b00657-f599-4593-a67c-4d138e0b0026 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:56.506918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19b00657-f599-4593-a67c-4d138e0b0026 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:56.507347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19b00657-f599-4593-a67c-4d138e0b0026 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:56.715765 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-19b00657-f599-4593-a67c-4d138e0b0026 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:56.725417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-19b00657-f599-4593-a67c-4d138e0b0026 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:56.726924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-19b00657-f599-4593-a67c-4d138e0b0026 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:56.728048 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-19b00657-f599-4593-a67c-4d138e0b0026 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1407 microversion: 2.20 time: 0.235554 Aug 07 01:14:56.728653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 321/617] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:14:56 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1407 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:14:57.751499 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-dd93a6f7-fd58-45f2-af08-7c7c3e4c627c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:57.752366 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-dd93a6f7-fd58-45f2-af08-7c7c3e4c627c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:57.759525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dd93a6f7-fd58-45f2-af08-7c7c3e4c627c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:57.760122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dd93a6f7-fd58-45f2-af08-7c7c3e4c627c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:57.760655 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dd93a6f7-fd58-45f2-af08-7c7c3e4c627c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:57.761373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dd93a6f7-fd58-45f2-af08-7c7c3e4c627c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:57.761822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dd93a6f7-fd58-45f2-af08-7c7c3e4c627c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:57.762247 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dd93a6f7-fd58-45f2-af08-7c7c3e4c627c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:58.028625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dd93a6f7-fd58-45f2-af08-7c7c3e4c627c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:58.038519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dd93a6f7-fd58-45f2-af08-7c7c3e4c627c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:58.040204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dd93a6f7-fd58-45f2-af08-7c7c3e4c627c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:58.041421 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-dd93a6f7-fd58-45f2-af08-7c7c3e4c627c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1407 microversion: 2.20 time: 0.294084 Aug 07 01:14:58.042026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 297/618] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:14:57 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1407 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:14:59.067984 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bc9d3dee-3fa3-40df-968a-aad90b447c05 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:14:59.068906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bc9d3dee-3fa3-40df-968a-aad90b447c05 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:14:59.078783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc9d3dee-3fa3-40df-968a-aad90b447c05 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:59.078783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc9d3dee-3fa3-40df-968a-aad90b447c05 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:59.078783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc9d3dee-3fa3-40df-968a-aad90b447c05 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:59.079736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc9d3dee-3fa3-40df-968a-aad90b447c05 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:59.080505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc9d3dee-3fa3-40df-968a-aad90b447c05 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:59.080731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc9d3dee-3fa3-40df-968a-aad90b447c05 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:59.247653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-bc9d3dee-3fa3-40df-968a-aad90b447c05 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:59.263096 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-bc9d3dee-3fa3-40df-968a-aad90b447c05 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:59.266898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-bc9d3dee-3fa3-40df-968a-aad90b447c05 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:14:59.269636 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bc9d3dee-3fa3-40df-968a-aad90b447c05 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1407 microversion: 2.20 time: 0.206048 Aug 07 01:14:59.271202 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 322/619] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:14:59 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1407 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:14:59.315423 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-fed624e3-095d-46fd-b3e2-d102a6b161ca req-3195d9c0-fe4c-41b4-80c9-c7c2439a47c3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "00b4795f-ad25-4565-bc64-10cbae94641a", "tag": "b2a8f175-ffc8-4e6e-a3f6-a939be50f0c9"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:59.336119 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-fed624e3-095d-46fd-b3e2-d102a6b161ca req-3195d9c0-fe4c-41b4-80c9-c7c2439a47c3 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:14:59.337233 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-fed624e3-095d-46fd-b3e2-d102a6b161ca req-3195d9c0-fe4c-41b4-80c9-c7c2439a47c3 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:14:59.338148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-fed624e3-095d-46fd-b3e2-d102a6b161ca req-3195d9c0-fe4c-41b4-80c9-c7c2439a47c3 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:14:59.361062 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-fed624e3-095d-46fd-b3e2-d102a6b161ca req-3195d9c0-fe4c-41b4-80c9-c7c2439a47c3 service nova] Creating event network-changed:b2a8f175-ffc8-4e6e-a3f6-a939be50f0c9 for instance 00b4795f-ad25-4565-bc64-10cbae94641a on hv-cinder-maste Aug 07 01:14:59.371089 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-fed624e3-095d-46fd-b3e2-d102a6b161ca req-3195d9c0-fe4c-41b4-80c9-c7c2439a47c3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072883 Aug 07 01:14:59.372883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 298/620] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:14:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:14:59.659131 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-207701737", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "networks": [{"uuid": "ea640be5-e5dd-479f-a491-fb3ae7cc114e"}]}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:14:59.745838 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.network.neutron [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] validate_networks() for [('ea640be5-e5dd-479f-a491-fb3ae7cc114e', None, None, None, None, None)] {{(pid=117983) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:15:00.264501 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Flavor limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:15:00.265194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Image limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:15:00.265486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Flavor pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:15:00.265916 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Image pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:15:00.267001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:15:00.277894 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:00.278410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:00.278851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:00.280576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:00.280576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:00.281114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:00.300672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Getting quotas for project f7b08cc6d4a34961bdf464b58946a02e. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:15:00.301511 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6853032d-90da-4131-89a8-aa354adb39ca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:00.303162 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6853032d-90da-4131-89a8-aa354adb39ca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:00.308544 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Getting quotas for user 6dd0ed2144164190b380ed58b3dd7cd4 and project f7b08cc6d4a34961bdf464b58946a02e. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:15:00.315710 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6853032d-90da-4131-89a8-aa354adb39ca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:00.316540 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6853032d-90da-4131-89a8-aa354adb39ca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:00.317337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6853032d-90da-4131-89a8-aa354adb39ca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:00.318620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6853032d-90da-4131-89a8-aa354adb39ca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:00.319482 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6853032d-90da-4131-89a8-aa354adb39ca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:00.320214 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6853032d-90da-4131-89a8-aa354adb39ca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:00.329695 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Going to run 1 instances... {{(pid=117983) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:15:00.332667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117983) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:15:00.383205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bb93f24d-58b3-41c6-9895-7817742a1dde tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.732183 Aug 07 01:15:00.384349 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 323/621] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:14:59 2022] POST /compute/v2.1/servers => generated 380 bytes in 734 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:15:00.406119 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ce1fd762-d9a7-434a-9ae7-fd8a5887c69d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:00.407928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ce1fd762-d9a7-434a-9ae7-fd8a5887c69d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:00.502606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ce1fd762-d9a7-434a-9ae7-fd8a5887c69d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:00.514652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ce1fd762-d9a7-434a-9ae7-fd8a5887c69d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1218 microversion: 2.1 time: 0.117688 Aug 07 01:15:00.515586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 324/622] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:00 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1218 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:00.521918 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6853032d-90da-4131-89a8-aa354adb39ca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:00.531521 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6853032d-90da-4131-89a8-aa354adb39ca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:00.534053 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6853032d-90da-4131-89a8-aa354adb39ca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:00.535354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6853032d-90da-4131-89a8-aa354adb39ca tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1407 microversion: 2.20 time: 0.242725 Aug 07 01:15:00.536056 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 299/623] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:15:00 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1407 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:15:01.552375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-136725a7-b2b9-4c3c-82f8-311d0dff90fa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:01.553374 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-136725a7-b2b9-4c3c-82f8-311d0dff90fa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:01.560015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-69e6fea6-0347-42b6-93df-93c126a10b89 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:01.561155 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-69e6fea6-0347-42b6-93df-93c126a10b89 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:01.565118 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-136725a7-b2b9-4c3c-82f8-311d0dff90fa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:01.566402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-136725a7-b2b9-4c3c-82f8-311d0dff90fa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:01.567100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-136725a7-b2b9-4c3c-82f8-311d0dff90fa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:01.573519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-136725a7-b2b9-4c3c-82f8-311d0dff90fa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:01.573519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-136725a7-b2b9-4c3c-82f8-311d0dff90fa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:01.573519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-136725a7-b2b9-4c3c-82f8-311d0dff90fa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:01.575599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69e6fea6-0347-42b6-93df-93c126a10b89 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:01.575599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69e6fea6-0347-42b6-93df-93c126a10b89 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:01.575599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69e6fea6-0347-42b6-93df-93c126a10b89 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:01.575904 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69e6fea6-0347-42b6-93df-93c126a10b89 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:01.576445 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69e6fea6-0347-42b6-93df-93c126a10b89 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:01.577375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69e6fea6-0347-42b6-93df-93c126a10b89 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:01.667713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-136725a7-b2b9-4c3c-82f8-311d0dff90fa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:01.682433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-136725a7-b2b9-4c3c-82f8-311d0dff90fa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1270 microversion: 2.1 time: 0.137819 Aug 07 01:15:01.683625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 325/624] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:01 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1270 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:01.885840 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-69e6fea6-0347-42b6-93df-93c126a10b89 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:01.909015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-69e6fea6-0347-42b6-93df-93c126a10b89 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:01.913738 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-69e6fea6-0347-42b6-93df-93c126a10b89 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:01.917350 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-69e6fea6-0347-42b6-93df-93c126a10b89 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1407 microversion: 2.20 time: 0.361889 Aug 07 01:15:01.918513 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 300/625] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:15:01 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1407 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:15:02.444891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a97cc170-98b3-4be7-85c9-fee7bd095c74 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:02.447415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a97cc170-98b3-4be7-85c9-fee7bd095c74 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:02.454984 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a97cc170-98b3-4be7-85c9-fee7bd095c74 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:02.455474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a97cc170-98b3-4be7-85c9-fee7bd095c74 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:02.455917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a97cc170-98b3-4be7-85c9-fee7bd095c74 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:02.456793 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a97cc170-98b3-4be7-85c9-fee7bd095c74 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:02.457156 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a97cc170-98b3-4be7-85c9-fee7bd095c74 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:02.457585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a97cc170-98b3-4be7-85c9-fee7bd095c74 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:02.492384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a97cc170-98b3-4be7-85c9-fee7bd095c74 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.052240 Aug 07 01:15:02.492986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 326/626] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:15:02 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903/os-volume_attachments => generated 25 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 07 01:15:02.563050 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6b76c327-786c-4aa2-adfd-062b33a50b7c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:02.564512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6b76c327-786c-4aa2-adfd-062b33a50b7c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:02.571902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b76c327-786c-4aa2-adfd-062b33a50b7c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:02.572539 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b76c327-786c-4aa2-adfd-062b33a50b7c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:02.573354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b76c327-786c-4aa2-adfd-062b33a50b7c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:02.574180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b76c327-786c-4aa2-adfd-062b33a50b7c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:02.574645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b76c327-786c-4aa2-adfd-062b33a50b7c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:02.575079 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b76c327-786c-4aa2-adfd-062b33a50b7c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:02.609493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6b76c327-786c-4aa2-adfd-062b33a50b7c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Going to try to terminate instance {{(pid=117984) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:15:02.719119 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6b76c327-786c-4aa2-adfd-062b33a50b7c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "DELETE /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 204 len: 0 microversion: 2.1 time: 0.159595 Aug 07 01:15:02.722578 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 301/627] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:15:02 2022] DELETE /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 07 01:15:02.722864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-26b419f6-18b8-43e4-9c31-81b600dc1dca tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:02.723630 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-26b419f6-18b8-43e4-9c31-81b600dc1dca tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:02.734336 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26b419f6-18b8-43e4-9c31-81b600dc1dca tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:02.735508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26b419f6-18b8-43e4-9c31-81b600dc1dca tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:02.738876 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-dda48076-9680-4642-9724-92a9948c8c41 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:02.738876 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-dda48076-9680-4642-9724-92a9948c8c41 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:02.739966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26b419f6-18b8-43e4-9c31-81b600dc1dca tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:02.741463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26b419f6-18b8-43e4-9c31-81b600dc1dca tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:02.742628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26b419f6-18b8-43e4-9c31-81b600dc1dca tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:02.743253 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26b419f6-18b8-43e4-9c31-81b600dc1dca tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:02.746961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dda48076-9680-4642-9724-92a9948c8c41 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:02.747662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dda48076-9680-4642-9724-92a9948c8c41 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:02.748309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dda48076-9680-4642-9724-92a9948c8c41 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:02.749484 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dda48076-9680-4642-9724-92a9948c8c41 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:02.751216 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dda48076-9680-4642-9724-92a9948c8c41 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:02.752549 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dda48076-9680-4642-9724-92a9948c8c41 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:02.996006 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dda48076-9680-4642-9724-92a9948c8c41 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:03.005902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-dda48076-9680-4642-9724-92a9948c8c41 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1674 microversion: 2.1 time: 0.276322 Aug 07 01:15:03.006702 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 302/628] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:02 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1674 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:03.017358 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3526d019-5f53-42a8-b87a-d8130b2c8f75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:03.018658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3526d019-5f53-42a8-b87a-d8130b2c8f75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:03.028557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3526d019-5f53-42a8-b87a-d8130b2c8f75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:03.029860 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3526d019-5f53-42a8-b87a-d8130b2c8f75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:03.030375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3526d019-5f53-42a8-b87a-d8130b2c8f75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:03.031301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3526d019-5f53-42a8-b87a-d8130b2c8f75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:03.032320 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3526d019-5f53-42a8-b87a-d8130b2c8f75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:03.032952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3526d019-5f53-42a8-b87a-d8130b2c8f75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:03.126918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-26b419f6-18b8-43e4-9c31-81b600dc1dca tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:03.136524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-26b419f6-18b8-43e4-9c31-81b600dc1dca tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1318 microversion: 2.1 time: 0.420326 Aug 07 01:15:03.142612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 327/629] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:02 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1318 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:03.156434 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:15:03.236027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3526d019-5f53-42a8-b87a-d8130b2c8f75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:03.246435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3526d019-5f53-42a8-b87a-d8130b2c8f75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:03.248168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3526d019-5f53-42a8-b87a-d8130b2c8f75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:03.249414 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3526d019-5f53-42a8-b87a-d8130b2c8f75 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1407 microversion: 2.20 time: 0.241419 Aug 07 01:15:03.250001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 303/630] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:15:03 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1407 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:15:04.030910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-45192439-a2f6-42dc-a35b-9108b1cbd6bf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:04.031616 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-45192439-a2f6-42dc-a35b-9108b1cbd6bf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:04.039326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45192439-a2f6-42dc-a35b-9108b1cbd6bf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:04.039822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45192439-a2f6-42dc-a35b-9108b1cbd6bf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:04.040292 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45192439-a2f6-42dc-a35b-9108b1cbd6bf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:04.041065 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45192439-a2f6-42dc-a35b-9108b1cbd6bf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:04.041556 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45192439-a2f6-42dc-a35b-9108b1cbd6bf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:04.042149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45192439-a2f6-42dc-a35b-9108b1cbd6bf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:04.161442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-528bd9ff-9704-4ca6-ac8d-1fcbd74c9285 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:04.162566 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-528bd9ff-9704-4ca6-ac8d-1fcbd74c9285 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:04.172296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-528bd9ff-9704-4ca6-ac8d-1fcbd74c9285 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:04.173126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-528bd9ff-9704-4ca6-ac8d-1fcbd74c9285 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:04.173839 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-528bd9ff-9704-4ca6-ac8d-1fcbd74c9285 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:04.174849 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-528bd9ff-9704-4ca6-ac8d-1fcbd74c9285 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:04.175429 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-528bd9ff-9704-4ca6-ac8d-1fcbd74c9285 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:04.175897 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-528bd9ff-9704-4ca6-ac8d-1fcbd74c9285 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:04.208141 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-45192439-a2f6-42dc-a35b-9108b1cbd6bf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:04.217591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-45192439-a2f6-42dc-a35b-9108b1cbd6bf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1674 microversion: 2.1 time: 0.190566 Aug 07 01:15:04.218491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 328/631] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:04 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1674 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:04.280535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-73386255-06e5-4f5d-b63b-497c6f2ab5e9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:04.282246 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-73386255-06e5-4f5d-b63b-497c6f2ab5e9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:04.292125 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73386255-06e5-4f5d-b63b-497c6f2ab5e9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:04.292860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73386255-06e5-4f5d-b63b-497c6f2ab5e9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:04.293486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73386255-06e5-4f5d-b63b-497c6f2ab5e9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:04.294705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73386255-06e5-4f5d-b63b-497c6f2ab5e9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:04.295326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73386255-06e5-4f5d-b63b-497c6f2ab5e9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:04.295844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73386255-06e5-4f5d-b63b-497c6f2ab5e9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:04.350842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-528bd9ff-9704-4ca6-ac8d-1fcbd74c9285 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:04.360784 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-528bd9ff-9704-4ca6-ac8d-1fcbd74c9285 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1318 microversion: 2.1 time: 0.204698 Aug 07 01:15:04.361589 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 304/632] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:04 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1318 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:04.510436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-73386255-06e5-4f5d-b63b-497c6f2ab5e9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:04.526092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-73386255-06e5-4f5d-b63b-497c6f2ab5e9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:04.529291 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-73386255-06e5-4f5d-b63b-497c6f2ab5e9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:04.531443 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-73386255-06e5-4f5d-b63b-497c6f2ab5e9 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1407 microversion: 2.20 time: 0.260522 Aug 07 01:15:04.532612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 329/633] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:15:04 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1407 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:15:05.107318 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-b3cd237f-7312-44be-ab8a-9f5fe319cd9c req-bf37fc07-6438-4c1f-bcb8-e5d140489a81 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "00b4795f-ad25-4565-bc64-10cbae94641a", "name": "network-vif-plugged", "status": "completed", "tag": "b2a8f175-ffc8-4e6e-a3f6-a939be50f0c9"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:15:05.129129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-b3cd237f-7312-44be-ab8a-9f5fe319cd9c req-bf37fc07-6438-4c1f-bcb8-e5d140489a81 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:05.130082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-b3cd237f-7312-44be-ab8a-9f5fe319cd9c req-bf37fc07-6438-4c1f-bcb8-e5d140489a81 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:05.130884 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-b3cd237f-7312-44be-ab8a-9f5fe319cd9c req-bf37fc07-6438-4c1f-bcb8-e5d140489a81 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:05.151743 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-b3cd237f-7312-44be-ab8a-9f5fe319cd9c req-bf37fc07-6438-4c1f-bcb8-e5d140489a81 service nova] Creating event network-vif-plugged:b2a8f175-ffc8-4e6e-a3f6-a939be50f0c9 for instance 00b4795f-ad25-4565-bc64-10cbae94641a on hv-cinder-maste Aug 07 01:15:05.161963 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-b3cd237f-7312-44be-ab8a-9f5fe319cd9c req-bf37fc07-6438-4c1f-bcb8-e5d140489a81 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060813 Aug 07 01:15:05.163244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 305/634] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:15:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:15:05.256491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-06264944-1c5d-4fc5-8631-bc5a1990b8f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:05.258384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-06264944-1c5d-4fc5-8631-bc5a1990b8f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:05.269871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-06264944-1c5d-4fc5-8631-bc5a1990b8f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:05.271222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-06264944-1c5d-4fc5-8631-bc5a1990b8f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:05.272294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-06264944-1c5d-4fc5-8631-bc5a1990b8f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:05.273920 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-06264944-1c5d-4fc5-8631-bc5a1990b8f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:05.280479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-06264944-1c5d-4fc5-8631-bc5a1990b8f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:05.286973 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-06264944-1c5d-4fc5-8631-bc5a1990b8f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:05.354155 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:15:05.391080 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ec92d1ea-e8c8-47f6-8e7a-d74f98b2e74d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:05.393442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ec92d1ea-e8c8-47f6-8e7a-d74f98b2e74d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:05.403373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec92d1ea-e8c8-47f6-8e7a-d74f98b2e74d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:05.404923 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec92d1ea-e8c8-47f6-8e7a-d74f98b2e74d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:05.405628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec92d1ea-e8c8-47f6-8e7a-d74f98b2e74d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:05.408071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec92d1ea-e8c8-47f6-8e7a-d74f98b2e74d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:05.408962 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec92d1ea-e8c8-47f6-8e7a-d74f98b2e74d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:05.409840 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ec92d1ea-e8c8-47f6-8e7a-d74f98b2e74d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:05.742836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-06264944-1c5d-4fc5-8631-bc5a1990b8f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:05.758432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-06264944-1c5d-4fc5-8631-bc5a1990b8f0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1674 microversion: 2.1 time: 0.508065 Aug 07 01:15:05.759654 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 330/635] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:05 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1674 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:05.765672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-aa41a2b5-ea80-4b75-9d51-58d83e6443fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:05.766956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-aa41a2b5-ea80-4b75-9d51-58d83e6443fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:05.774852 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa41a2b5-ea80-4b75-9d51-58d83e6443fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:05.775960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa41a2b5-ea80-4b75-9d51-58d83e6443fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:05.776656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa41a2b5-ea80-4b75-9d51-58d83e6443fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:05.777663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa41a2b5-ea80-4b75-9d51-58d83e6443fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:05.778857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa41a2b5-ea80-4b75-9d51-58d83e6443fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:05.780038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa41a2b5-ea80-4b75-9d51-58d83e6443fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:06.007798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ec92d1ea-e8c8-47f6-8e7a-d74f98b2e74d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:06.016819 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ec92d1ea-e8c8-47f6-8e7a-d74f98b2e74d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1318 microversion: 2.1 time: 0.635983 Aug 07 01:15:06.031013 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 306/636] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:05 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1318 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:06.213204 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-aa41a2b5-ea80-4b75-9d51-58d83e6443fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:06.228444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-aa41a2b5-ea80-4b75-9d51-58d83e6443fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:06.231559 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-aa41a2b5-ea80-4b75-9d51-58d83e6443fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:06.233794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-aa41a2b5-ea80-4b75-9d51-58d83e6443fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1407 microversion: 2.20 time: 0.472403 Aug 07 01:15:06.235036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 331/637] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:15:05 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1407 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:15:06.262810 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-1af83781-94c0-4663-99c7-dd95624307c9 req-7cdcee0b-1a80-4a75-a003-459bd6a254e7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f481f4cc-2803-4f12-9cd2-db6055d1488c", "tag": "77812789-b59f-4abe-9e20-b16f511c9fe2"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:15:06.296854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-1af83781-94c0-4663-99c7-dd95624307c9 req-7cdcee0b-1a80-4a75-a003-459bd6a254e7 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:06.297346 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-1af83781-94c0-4663-99c7-dd95624307c9 req-7cdcee0b-1a80-4a75-a003-459bd6a254e7 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:06.297783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-1af83781-94c0-4663-99c7-dd95624307c9 req-7cdcee0b-1a80-4a75-a003-459bd6a254e7 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:06.318434 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-1af83781-94c0-4663-99c7-dd95624307c9 req-7cdcee0b-1a80-4a75-a003-459bd6a254e7 service nova] Creating event network-changed:77812789-b59f-4abe-9e20-b16f511c9fe2 for instance f481f4cc-2803-4f12-9cd2-db6055d1488c on hv-cinder-maste Aug 07 01:15:06.324793 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-1af83781-94c0-4663-99c7-dd95624307c9 req-7cdcee0b-1a80-4a75-a003-459bd6a254e7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.075326 Aug 07 01:15:06.326105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 307/638] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:15:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:15:06.801437 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7e991abc-63ec-4826-b7a5-ab727bdb6fc0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:06.802217 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7e991abc-63ec-4826-b7a5-ab727bdb6fc0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:06.815092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e991abc-63ec-4826-b7a5-ab727bdb6fc0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:06.816048 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e991abc-63ec-4826-b7a5-ab727bdb6fc0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:06.816910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e991abc-63ec-4826-b7a5-ab727bdb6fc0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:06.817945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e991abc-63ec-4826-b7a5-ab727bdb6fc0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:06.820675 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e991abc-63ec-4826-b7a5-ab727bdb6fc0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:06.822183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e991abc-63ec-4826-b7a5-ab727bdb6fc0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:06.861969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-c685d448-9a9a-4934-be0d-f9675d5c6d22 req-f74ad823-d3a8-49ca-8908-4855e8677482 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4f3f0774-3a34-4f19-9233-c4e55c7f5903", "name": "network-vif-deleted", "tag": "9e5aa5e7-49cb-4c56-b9f5-3adf7c37f16a"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:15:06.897287 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c685d448-9a9a-4934-be0d-f9675d5c6d22 req-f74ad823-d3a8-49ca-8908-4855e8677482 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:06.898066 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c685d448-9a9a-4934-be0d-f9675d5c6d22 req-f74ad823-d3a8-49ca-8908-4855e8677482 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:06.898901 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c685d448-9a9a-4934-be0d-f9675d5c6d22 req-f74ad823-d3a8-49ca-8908-4855e8677482 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:06.934721 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-c685d448-9a9a-4934-be0d-f9675d5c6d22 req-f74ad823-d3a8-49ca-8908-4855e8677482 service nova] Creating event network-vif-deleted:9e5aa5e7-49cb-4c56-b9f5-3adf7c37f16a for instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 on hv-cinder-maste Aug 07 01:15:06.947804 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-c685d448-9a9a-4934-be0d-f9675d5c6d22 req-f74ad823-d3a8-49ca-8908-4855e8677482 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.098584 Aug 07 01:15:06.956689 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 308/639] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:15:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:15:07.010137 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7e991abc-63ec-4826-b7a5-ab727bdb6fc0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:07.025942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7e991abc-63ec-4826-b7a5-ab727bdb6fc0 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1607 microversion: 2.1 time: 0.233127 Aug 07 01:15:07.040576 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9a62f148-e6bc-45ac-be71-fb3b1c750fa2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:07.041452 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9a62f148-e6bc-45ac-be71-fb3b1c750fa2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:07.056900 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a62f148-e6bc-45ac-be71-fb3b1c750fa2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:07.057195 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a62f148-e6bc-45ac-be71-fb3b1c750fa2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:07.057958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a62f148-e6bc-45ac-be71-fb3b1c750fa2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:07.058941 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a62f148-e6bc-45ac-be71-fb3b1c750fa2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:07.059910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 332/640] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:06 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1607 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:07.065155 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a62f148-e6bc-45ac-be71-fb3b1c750fa2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:07.065805 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a62f148-e6bc-45ac-be71-fb3b1c750fa2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:07.275496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a44312fb-1aef-43e0-b7e9-e175f33cd2fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:07.276544 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a44312fb-1aef-43e0-b7e9-e175f33cd2fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:07.290738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a44312fb-1aef-43e0-b7e9-e175f33cd2fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:07.291644 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a44312fb-1aef-43e0-b7e9-e175f33cd2fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:07.292853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a44312fb-1aef-43e0-b7e9-e175f33cd2fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:07.293708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a44312fb-1aef-43e0-b7e9-e175f33cd2fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:07.294539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a44312fb-1aef-43e0-b7e9-e175f33cd2fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:07.295293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a44312fb-1aef-43e0-b7e9-e175f33cd2fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:07.311286 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9a62f148-e6bc-45ac-be71-fb3b1c750fa2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:07.325952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9a62f148-e6bc-45ac-be71-fb3b1c750fa2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1318 microversion: 2.1 time: 0.289856 Aug 07 01:15:07.327380 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 309/641] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:07 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1318 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:07.523019 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a44312fb-1aef-43e0-b7e9-e175f33cd2fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:07.532594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a44312fb-1aef-43e0-b7e9-e175f33cd2fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:07.534146 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a44312fb-1aef-43e0-b7e9-e175f33cd2fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:07.535604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a44312fb-1aef-43e0-b7e9-e175f33cd2fe tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1407 microversion: 2.20 time: 0.269058 Aug 07 01:15:07.536347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 333/642] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:15:07 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1407 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:15:08.052023 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-16b4f7e9-61c4-4aa5-ba65-b61955b4b9ac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:08.053042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-16b4f7e9-61c4-4aa5-ba65-b61955b4b9ac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:08.064123 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16b4f7e9-61c4-4aa5-ba65-b61955b4b9ac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:08.065187 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16b4f7e9-61c4-4aa5-ba65-b61955b4b9ac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:08.065838 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16b4f7e9-61c4-4aa5-ba65-b61955b4b9ac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:08.066764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16b4f7e9-61c4-4aa5-ba65-b61955b4b9ac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:08.067676 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16b4f7e9-61c4-4aa5-ba65-b61955b4b9ac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:08.068739 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16b4f7e9-61c4-4aa5-ba65-b61955b4b9ac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:08.145669 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16b4f7e9-61c4-4aa5-ba65-b61955b4b9ac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:08.146890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16b4f7e9-61c4-4aa5-ba65-b61955b4b9ac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:08.148205 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16b4f7e9-61c4-4aa5-ba65-b61955b4b9ac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:08.149788 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.objects.instance [None req-16b4f7e9-61c4-4aa5-ba65-b61955b4b9ac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lazy-loading 'fault' on Instance uuid 4f3f0774-3a34-4f19-9233-c4e55c7f5903 {{(pid=117984) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:15:08.244450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-16b4f7e9-61c4-4aa5-ba65-b61955b4b9ac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:08.254776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-16b4f7e9-61c4-4aa5-ba65-b61955b4b9ac tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 200 len: 1328 microversion: 2.1 time: 0.206285 Aug 07 01:15:08.268911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 310/643] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:08 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 1328 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:08.364487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d020d9f0-bf55-4d43-bb57-36004a9888b2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:08.365386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d020d9f0-bf55-4d43-bb57-36004a9888b2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:08.368201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=1,nova_api:INSERT=7 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:15:08.376805 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d020d9f0-bf55-4d43-bb57-36004a9888b2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:08.377524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d020d9f0-bf55-4d43-bb57-36004a9888b2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:08.378037 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d020d9f0-bf55-4d43-bb57-36004a9888b2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:08.378897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d020d9f0-bf55-4d43-bb57-36004a9888b2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:08.379434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d020d9f0-bf55-4d43-bb57-36004a9888b2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:08.380024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d020d9f0-bf55-4d43-bb57-36004a9888b2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:08.575289 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-275afe8b-8d00-4657-8f8c-1a4fb76ec38a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:08.576439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-275afe8b-8d00-4657-8f8c-1a4fb76ec38a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:08.576852 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d020d9f0-bf55-4d43-bb57-36004a9888b2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:08.587290 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-275afe8b-8d00-4657-8f8c-1a4fb76ec38a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:08.588334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-275afe8b-8d00-4657-8f8c-1a4fb76ec38a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:08.589214 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-275afe8b-8d00-4657-8f8c-1a4fb76ec38a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:08.590528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-275afe8b-8d00-4657-8f8c-1a4fb76ec38a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:08.590750 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d020d9f0-bf55-4d43-bb57-36004a9888b2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1318 microversion: 2.1 time: 0.232788 Aug 07 01:15:08.591313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-275afe8b-8d00-4657-8f8c-1a4fb76ec38a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:08.591424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 334/644] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:08 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1318 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:08.592071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-275afe8b-8d00-4657-8f8c-1a4fb76ec38a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:08.835748 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-275afe8b-8d00-4657-8f8c-1a4fb76ec38a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:08.856441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-275afe8b-8d00-4657-8f8c-1a4fb76ec38a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:08.859583 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-275afe8b-8d00-4657-8f8c-1a4fb76ec38a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:08.862053 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-275afe8b-8d00-4657-8f8c-1a4fb76ec38a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1600 microversion: 2.20 time: 0.293996 Aug 07 01:15:08.863148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 311/645] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:15:08 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1600 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:15:08.890974 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-101cd51d-4b8d-43cd-9d52-cf5b73fce397 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:08.892408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-101cd51d-4b8d-43cd-9d52-cf5b73fce397 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:08.902104 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-101cd51d-4b8d-43cd-9d52-cf5b73fce397 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:08.903070 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-101cd51d-4b8d-43cd-9d52-cf5b73fce397 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:08.903855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-101cd51d-4b8d-43cd-9d52-cf5b73fce397 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:08.905190 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-101cd51d-4b8d-43cd-9d52-cf5b73fce397 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:08.906046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-101cd51d-4b8d-43cd-9d52-cf5b73fce397 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:08.906866 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-101cd51d-4b8d-43cd-9d52-cf5b73fce397 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:09.029036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-101cd51d-4b8d-43cd-9d52-cf5b73fce397 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.145506 Aug 07 01:15:09.029694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 335/646] 10.222.0.26 () {60 vars in 1316 bytes} [Sun Aug 7 01:15:08 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a/os-interface => generated 285 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:15:09.288157 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1436eb7b-ccb7-42ce-af50-b84c1f9525a6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:09.289589 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1436eb7b-ccb7-42ce-af50-b84c1f9525a6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:09.301728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1436eb7b-ccb7-42ce-af50-b84c1f9525a6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:09.302826 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1436eb7b-ccb7-42ce-af50-b84c1f9525a6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:09.303611 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1436eb7b-ccb7-42ce-af50-b84c1f9525a6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:09.305021 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1436eb7b-ccb7-42ce-af50-b84c1f9525a6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:09.306074 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1436eb7b-ccb7-42ce-af50-b84c1f9525a6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:09.306816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1436eb7b-ccb7-42ce-af50-b84c1f9525a6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:09.336837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-1436eb7b-ccb7-42ce-af50-b84c1f9525a6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] HTTP exception thrown: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1436eb7b-ccb7-42ce-af50-b84c1f9525a6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Request method failure captured: Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 HTTP/1.1 Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xD9cznP8hWf4pmxdU3hNl0ZKgrnQCZpWvAOumCUzV3uHmYQXZPbB6q2YtbZ5VRmM0fulvc4-ky89A5e1E7XmdwEvMsFQ8K0SJLQxjef3QqDDgPjeGMqBD9ErxCwISYFPrZdE0TGsrQI6E1EI8ML9K90t4_XH8zTJg5NaAH5XIw5714 Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 7fd9ab6d98a44605a57bd8ae9535a686 Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:15:09.340110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:15:09.342262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:15:09.342262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:15:09.342262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 7fd9ab6d98a44605a57bd8ae9535a686 Aug 07 01:15:09.342262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:15:09.342262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-AttachVolumeTestJSON-1659395094-project Aug 07 01:15:09.342262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:15:09.342262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:15:09.342262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: 4ae9c7ef17b746fc8f87e3c22a241b76 Aug 07 01:15:09.342262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-AttachVolumeTestJSON-1659395094-project Aug 07 01:15:09.342262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:15:09.342262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:15:09.342262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '4f3f0774-3a34-4f19-9233-c4e55c7f5903'} Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:15:09.343293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:15:09.346467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:15:09.349213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1436eb7b-ccb7-42ce-af50-b84c1f9525a6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Returning 404 to user: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:15:09.349213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1436eb7b-ccb7-42ce-af50-b84c1f9525a6 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 404 len: 111 microversion: 2.1 time: 0.062402 Aug 07 01:15:09.349213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 312/647] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:09 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:15:09.358837 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2025a3bc-2148-4171-9281-cea3a8f66620 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '.version_select at 0x7f1fad29fb80>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:09.393168 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2025a3bc-2148-4171-9281-cea3a8f66620 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1012330668" status: 202 len: 0 microversion: 2.1 time: 0.041283 Aug 07 01:15:09.394234 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 336/648] 10.222.0.26 () {58 vars in 1221 bytes} [Sun Aug 7 01:15:09 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1012330668 => generated 0 bytes in 44 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:15:09.619219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ee855c67-51f3-4368-8bbb-bcbb067dfc32 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:09.620954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ee855c67-51f3-4368-8bbb-bcbb067dfc32 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:09.628120 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ee855c67-51f3-4368-8bbb-bcbb067dfc32 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:09.629125 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ee855c67-51f3-4368-8bbb-bcbb067dfc32 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:09.629788 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ee855c67-51f3-4368-8bbb-bcbb067dfc32 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:09.630787 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ee855c67-51f3-4368-8bbb-bcbb067dfc32 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:09.631843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ee855c67-51f3-4368-8bbb-bcbb067dfc32 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:09.632600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ee855c67-51f3-4368-8bbb-bcbb067dfc32 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:09.661504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:15:10.237442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ee855c67-51f3-4368-8bbb-bcbb067dfc32 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:10.260666 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ee855c67-51f3-4368-8bbb-bcbb067dfc32 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1318 microversion: 2.1 time: 0.643559 Aug 07 01:15:10.262327 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 313/649] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:09 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1318 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:10.761475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=4 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:15:10.765258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=18 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:15:10.776538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ffbc2d40-aa1f-4db2-8fa0-f23c83689faf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Action: 'create', calling method: .version_select at 0x7f1fad23ee50>, body: {"keypair": {"name": "tempest-keypair-2048332820", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGBjUkjzHsm+PJNqSd1NabmfS9HkV3Hisu8y5G4pJl7BW2px0joRE1dg+Z+ML3zJ+ZTi5K+ixkY3r8+DLF0BCg/rgZ3JUDeSulpu1EpEuBhdgcrPdL65jJT2JeNeYvHD1w=="}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:15:10.806653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-ffbc2d40-aa1f-4db2-8fa0-f23c83689faf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Getting quotas for project 7fd9ab6d98a44605a57bd8ae9535a686. Resources: {'key_pairs'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:15:10.816378 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-ffbc2d40-aa1f-4db2-8fa0-f23c83689faf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Getting quotas for user 4ae9c7ef17b746fc8f87e3c22a241b76 and project 7fd9ab6d98a44605a57bd8ae9535a686. Resources: {'key_pairs'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:15:10.840983 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-29601550-737e-422f-b2b4-20f8f575f0d5 req-d3866d75-7fa5-41ce-ad9c-cb5e63d69040 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "00b4795f-ad25-4565-bc64-10cbae94641a", "tag": "b2a8f175-ffc8-4e6e-a3f6-a939be50f0c9"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:15:10.858508 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-29601550-737e-422f-b2b4-20f8f575f0d5 req-d3866d75-7fa5-41ce-ad9c-cb5e63d69040 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:10.859299 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-29601550-737e-422f-b2b4-20f8f575f0d5 req-d3866d75-7fa5-41ce-ad9c-cb5e63d69040 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:10.860054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-29601550-737e-422f-b2b4-20f8f575f0d5 req-d3866d75-7fa5-41ce-ad9c-cb5e63d69040 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:10.868670 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ffbc2d40-aa1f-4db2-8fa0-f23c83689faf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.108803 Aug 07 01:15:10.870269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 337/650] 10.222.0.26 () {60 vars in 1159 bytes} [Sun Aug 7 01:15:10 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:15:10.880161 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-29601550-737e-422f-b2b4-20f8f575f0d5 req-d3866d75-7fa5-41ce-ad9c-cb5e63d69040 service nova] Creating event network-changed:b2a8f175-ffc8-4e6e-a3f6-a939be50f0c9 for instance 00b4795f-ad25-4565-bc64-10cbae94641a on hv-cinder-maste Aug 07 01:15:10.889300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-29601550-737e-422f-b2b4-20f8f575f0d5 req-d3866d75-7fa5-41ce-ad9c-cb5e63d69040 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052503 Aug 07 01:15:10.890911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 314/651] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:15:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:15:11.304201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e580c604-30a3-47a8-bdab-0b9f367796e0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:11.305301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e580c604-30a3-47a8-bdab-0b9f367796e0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:11.319358 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e580c604-30a3-47a8-bdab-0b9f367796e0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:11.320190 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e580c604-30a3-47a8-bdab-0b9f367796e0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:11.321070 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e580c604-30a3-47a8-bdab-0b9f367796e0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:11.322418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e580c604-30a3-47a8-bdab-0b9f367796e0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:11.323257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e580c604-30a3-47a8-bdab-0b9f367796e0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:11.328926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e580c604-30a3-47a8-bdab-0b9f367796e0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:11.768139 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e580c604-30a3-47a8-bdab-0b9f367796e0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:11.803350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e580c604-30a3-47a8-bdab-0b9f367796e0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1318 microversion: 2.1 time: 0.506281 Aug 07 01:15:11.806955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 338/652] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:11 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1318 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:12.852410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-28aebb43-65a8-4425-8d97-551f33e4ac8f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:12.854353 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-28aebb43-65a8-4425-8d97-551f33e4ac8f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:12.871718 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-28aebb43-65a8-4425-8d97-551f33e4ac8f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:12.872845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-28aebb43-65a8-4425-8d97-551f33e4ac8f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:12.873791 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-28aebb43-65a8-4425-8d97-551f33e4ac8f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:12.875080 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-28aebb43-65a8-4425-8d97-551f33e4ac8f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:12.876003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-28aebb43-65a8-4425-8d97-551f33e4ac8f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:12.876764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-28aebb43-65a8-4425-8d97-551f33e4ac8f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:13.220659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-28aebb43-65a8-4425-8d97-551f33e4ac8f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:13.238684 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-28aebb43-65a8-4425-8d97-551f33e4ac8f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1318 microversion: 2.1 time: 0.394669 Aug 07 01:15:13.239765 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 315/653] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:12 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1318 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:13.370445 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-778927675", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "b1257e6a-7a97-43f5-969f-654d75e2ab7d"}], "security_groups": [{"name": "tempest-securitygroup--211122002"}], "key_name": "tempest-keypair-2048332820", "user_data": "}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:15:13.530301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.network.neutron [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] validate_networks() for [('b1257e6a-7a97-43f5-969f-654d75e2ab7d', None, None, None, None, None)] {{(pid=117983) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:15:13.765250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-4586f3ac-135a-48a9-9059-6a602d96fb47 req-5d4408be-c200-4660-97ad-f2f595baefcd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f481f4cc-2803-4f12-9cd2-db6055d1488c", "name": "network-vif-plugged", "status": "completed", "tag": "77812789-b59f-4abe-9e20-b16f511c9fe2"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:15:13.780681 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=74,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:15:13.789754 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-4586f3ac-135a-48a9-9059-6a602d96fb47 req-5d4408be-c200-4660-97ad-f2f595baefcd service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:13.790941 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-4586f3ac-135a-48a9-9059-6a602d96fb47 req-5d4408be-c200-4660-97ad-f2f595baefcd service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:13.791911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-4586f3ac-135a-48a9-9059-6a602d96fb47 req-5d4408be-c200-4660-97ad-f2f595baefcd service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:13.822792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-4586f3ac-135a-48a9-9059-6a602d96fb47 req-5d4408be-c200-4660-97ad-f2f595baefcd service nova] Creating event network-vif-plugged:77812789-b59f-4abe-9e20-b16f511c9fe2 for instance f481f4cc-2803-4f12-9cd2-db6055d1488c on hv-cinder-maste Aug 07 01:15:13.835160 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-4586f3ac-135a-48a9-9059-6a602d96fb47 req-5d4408be-c200-4660-97ad-f2f595baefcd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.077262 Aug 07 01:15:13.836578 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 316/654] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:15:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:15:14.279697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-463641e3-0ead-4eaa-8912-288b514f0932 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:14.282453 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-463641e3-0ead-4eaa-8912-288b514f0932 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:14.294553 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-463641e3-0ead-4eaa-8912-288b514f0932 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:14.295151 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-463641e3-0ead-4eaa-8912-288b514f0932 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:14.295660 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-463641e3-0ead-4eaa-8912-288b514f0932 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:14.296728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-463641e3-0ead-4eaa-8912-288b514f0932 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:14.298089 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-463641e3-0ead-4eaa-8912-288b514f0932 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:14.298591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-463641e3-0ead-4eaa-8912-288b514f0932 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:14.306042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Flavor limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:15:14.306908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Image limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:15:14.308705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Flavor pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:15:14.310269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Image pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:15:14.311267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:15:14.330705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:14.332051 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:14.332981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:14.334417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:14.335758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:14.336650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:14.369804 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Getting quotas for project 7fd9ab6d98a44605a57bd8ae9535a686. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:15:14.375321 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Getting quotas for user 4ae9c7ef17b746fc8f87e3c22a241b76 and project 7fd9ab6d98a44605a57bd8ae9535a686. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:15:14.392147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Going to run 1 instances... {{(pid=117983) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:15:14.394992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117983) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:15:14.472080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5d1fc202-727b-4984-b3d3-9e19fcde0772 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.105850 Aug 07 01:15:14.473446 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 339/655] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:15:13 2022] POST /compute/v2.1/servers => generated 401 bytes in 1109 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:15:14.490540 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7411739f-d65d-4a73-9ab9-1e7943c8f55c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:14.491983 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7411739f-d65d-4a73-9ab9-1e7943c8f55c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:14.555243 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-463641e3-0ead-4eaa-8912-288b514f0932 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:14.576671 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-463641e3-0ead-4eaa-8912-288b514f0932 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1318 microversion: 2.1 time: 0.304875 Aug 07 01:15:14.577888 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 317/656] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:14 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1318 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:14.604151 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7411739f-d65d-4a73-9ab9-1e7943c8f55c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:14.611953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7411739f-d65d-4a73-9ab9-1e7943c8f55c tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1237 microversion: 2.1 time: 0.129874 Aug 07 01:15:14.612596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 340/657] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:14 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1237 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:15.615557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2964998b-ee88-44be-9ca0-47d99e50f290 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:15.616817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2964998b-ee88-44be-9ca0-47d99e50f290 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:15.627578 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2964998b-ee88-44be-9ca0-47d99e50f290 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:15.628527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2964998b-ee88-44be-9ca0-47d99e50f290 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:15.629130 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2964998b-ee88-44be-9ca0-47d99e50f290 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:15.630051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2964998b-ee88-44be-9ca0-47d99e50f290 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:15.630654 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2964998b-ee88-44be-9ca0-47d99e50f290 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:15.631865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2964998b-ee88-44be-9ca0-47d99e50f290 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:15.649698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6d36ad8f-b379-4862-890f-8c495f949100 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:15.650817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6d36ad8f-b379-4862-890f-8c495f949100 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:15.663221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d36ad8f-b379-4862-890f-8c495f949100 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:15.664137 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d36ad8f-b379-4862-890f-8c495f949100 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:15.665284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d36ad8f-b379-4862-890f-8c495f949100 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:15.666903 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d36ad8f-b379-4862-890f-8c495f949100 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:15.667779 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d36ad8f-b379-4862-890f-8c495f949100 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:15.668854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d36ad8f-b379-4862-890f-8c495f949100 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:15.792887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6d36ad8f-b379-4862-890f-8c495f949100 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:15.807578 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6d36ad8f-b379-4862-890f-8c495f949100 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1289 microversion: 2.1 time: 0.162244 Aug 07 01:15:15.808697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 341/658] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:15 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1289 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:15.833312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2964998b-ee88-44be-9ca0-47d99e50f290 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:15.846605 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2964998b-ee88-44be-9ca0-47d99e50f290 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1318 microversion: 2.1 time: 0.237123 Aug 07 01:15:15.847651 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 318/659] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:15 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1318 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:16.839732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1f986701-f9f0-4c1b-96e6-adbf1e45bcdf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:16.840399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1f986701-f9f0-4c1b-96e6-adbf1e45bcdf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:16.851603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1f986701-f9f0-4c1b-96e6-adbf1e45bcdf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:16.852511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1f986701-f9f0-4c1b-96e6-adbf1e45bcdf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:16.853282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1f986701-f9f0-4c1b-96e6-adbf1e45bcdf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:16.854555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1f986701-f9f0-4c1b-96e6-adbf1e45bcdf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:16.855368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1f986701-f9f0-4c1b-96e6-adbf1e45bcdf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:16.856114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1f986701-f9f0-4c1b-96e6-adbf1e45bcdf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:16.881543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1ca87216-9e4d-42b5-9798-464a4cb6dd93 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:16.882239 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1ca87216-9e4d-42b5-9798-464a4cb6dd93 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:16.890522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ca87216-9e4d-42b5-9798-464a4cb6dd93 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:16.891932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ca87216-9e4d-42b5-9798-464a4cb6dd93 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:16.892611 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ca87216-9e4d-42b5-9798-464a4cb6dd93 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:16.893614 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ca87216-9e4d-42b5-9798-464a4cb6dd93 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:16.894083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ca87216-9e4d-42b5-9798-464a4cb6dd93 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:16.894571 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ca87216-9e4d-42b5-9798-464a4cb6dd93 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:16.986613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1f986701-f9f0-4c1b-96e6-adbf1e45bcdf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:16.999115 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1f986701-f9f0-4c1b-96e6-adbf1e45bcdf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1295 microversion: 2.1 time: 0.162767 Aug 07 01:15:17.002877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 342/660] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:16 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1295 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:17.089553 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1ca87216-9e4d-42b5-9798-464a4cb6dd93 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:17.098988 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1ca87216-9e4d-42b5-9798-464a4cb6dd93 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1508 microversion: 2.1 time: 0.221528 Aug 07 01:15:17.099704 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 319/661] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:16 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1508 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:17.121926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-332732cb-c131-4186-b727-23ade0c4d5e4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "830234fe-bbd1-4781-97ce-8e340a8d9518", "device": "/dev/sdb"}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:15:17.125125 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-332732cb-c131-4186-b727-23ade0c4d5e4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:17.132694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-332732cb-c131-4186-b727-23ade0c4d5e4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:17.133194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-332732cb-c131-4186-b727-23ade0c4d5e4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:17.133630 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-332732cb-c131-4186-b727-23ade0c4d5e4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:17.134590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-332732cb-c131-4186-b727-23ade0c4d5e4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:17.135058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-332732cb-c131-4186-b727-23ade0c4d5e4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:17.135490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-332732cb-c131-4186-b727-23ade0c4d5e4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:17.487792 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-332732cb-c131-4186-b727-23ade0c4d5e4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "POST /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.370187 Aug 07 01:15:17.488789 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 343/662] 10.222.0.26 () {60 vars in 1323 bytes} [Sun Aug 7 01:15:17 2022] POST /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments => generated 194 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:15:18.042774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-af79ed98-4e3c-440a-9758-b8f7385c277e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:18.045168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-af79ed98-4e3c-440a-9758-b8f7385c277e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:18.059561 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af79ed98-4e3c-440a-9758-b8f7385c277e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:18.060557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af79ed98-4e3c-440a-9758-b8f7385c277e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:18.061375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af79ed98-4e3c-440a-9758-b8f7385c277e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:18.062765 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af79ed98-4e3c-440a-9758-b8f7385c277e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:18.063857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af79ed98-4e3c-440a-9758-b8f7385c277e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:18.064662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af79ed98-4e3c-440a-9758-b8f7385c277e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:18.315839 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-af79ed98-4e3c-440a-9758-b8f7385c277e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:18.333199 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-af79ed98-4e3c-440a-9758-b8f7385c277e tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1362 microversion: 2.1 time: 0.298494 Aug 07 01:15:18.334712 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 320/663] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:18 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1362 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:18.974662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-d683148f-83c8-4208-9d52-81733e00cd67 req-217c214a-d874-4b60-9178-bdd7e71a707f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8e8c48f1-19d6-4691-9e54-e5d84962485a", "tag": "cca96b0c-1343-4c42-8e76-9d5bf2321cee"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:15:18.997090 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-d683148f-83c8-4208-9d52-81733e00cd67 req-217c214a-d874-4b60-9178-bdd7e71a707f service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:18.998487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-d683148f-83c8-4208-9d52-81733e00cd67 req-217c214a-d874-4b60-9178-bdd7e71a707f service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:18.999790 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-d683148f-83c8-4208-9d52-81733e00cd67 req-217c214a-d874-4b60-9178-bdd7e71a707f service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:19.026912 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-d683148f-83c8-4208-9d52-81733e00cd67 req-217c214a-d874-4b60-9178-bdd7e71a707f service nova] Creating event network-changed:cca96b0c-1343-4c42-8e76-9d5bf2321cee for instance 8e8c48f1-19d6-4691-9e54-e5d84962485a on hv-cinder-maste Aug 07 01:15:19.037385 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-d683148f-83c8-4208-9d52-81733e00cd67 req-217c214a-d874-4b60-9178-bdd7e71a707f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069563 Aug 07 01:15:19.039032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 344/664] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:15:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:15:19.363435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e4a9da1c-a841-46c7-bb4f-c0d0887bc61f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:19.364672 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e4a9da1c-a841-46c7-bb4f-c0d0887bc61f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:19.373601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4a9da1c-a841-46c7-bb4f-c0d0887bc61f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:19.374229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4a9da1c-a841-46c7-bb4f-c0d0887bc61f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:19.374786 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4a9da1c-a841-46c7-bb4f-c0d0887bc61f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:19.375862 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4a9da1c-a841-46c7-bb4f-c0d0887bc61f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:19.376564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4a9da1c-a841-46c7-bb4f-c0d0887bc61f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:19.377266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4a9da1c-a841-46c7-bb4f-c0d0887bc61f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:19.595764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e4a9da1c-a841-46c7-bb4f-c0d0887bc61f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:19.613123 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e4a9da1c-a841-46c7-bb4f-c0d0887bc61f tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1362 microversion: 2.1 time: 0.257444 Aug 07 01:15:19.614398 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 321/665] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:19 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1362 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:20.357718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3f59f5ff-0edc-47f3-8c0f-ab16761f959d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "49c52ecb-fd7f-409d-b4c7-6f5bb414da0f"}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:15:20.360283 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3f59f5ff-0edc-47f3-8c0f-ab16761f959d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:20.367391 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f59f5ff-0edc-47f3-8c0f-ab16761f959d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:20.367894 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f59f5ff-0edc-47f3-8c0f-ab16761f959d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:20.368381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f59f5ff-0edc-47f3-8c0f-ab16761f959d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:20.369116 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f59f5ff-0edc-47f3-8c0f-ab16761f959d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:20.369581 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f59f5ff-0edc-47f3-8c0f-ab16761f959d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:20.370015 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f59f5ff-0edc-47f3-8c0f-ab16761f959d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:20.637021 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5c08f9a0-f8e8-4e01-b60c-9c66b901dfaf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:20.637951 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5c08f9a0-f8e8-4e01-b60c-9c66b901dfaf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:20.646236 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c08f9a0-f8e8-4e01-b60c-9c66b901dfaf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:20.646897 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c08f9a0-f8e8-4e01-b60c-9c66b901dfaf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:20.647444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c08f9a0-f8e8-4e01-b60c-9c66b901dfaf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:20.648324 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c08f9a0-f8e8-4e01-b60c-9c66b901dfaf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:20.648886 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c08f9a0-f8e8-4e01-b60c-9c66b901dfaf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:20.649418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c08f9a0-f8e8-4e01-b60c-9c66b901dfaf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:20.908222 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5c08f9a0-f8e8-4e01-b60c-9c66b901dfaf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:20.927566 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5c08f9a0-f8e8-4e01-b60c-9c66b901dfaf tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1362 microversion: 2.1 time: 0.294371 Aug 07 01:15:20.930641 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 322/666] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:20 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1362 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:21.200403 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3f59f5ff-0edc-47f3-8c0f-ab16761f959d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "POST /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.848027 Aug 07 01:15:21.201044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 345/667] 10.222.0.26 () {60 vars in 1323 bytes} [Sun Aug 7 01:15:20 2022] POST /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 194 bytes in 850 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:15:22.035258 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-51507cb3-c214-4322-8156-a0c2c7b408d3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:22.036147 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-51507cb3-c214-4322-8156-a0c2c7b408d3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:22.043350 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-51507cb3-c214-4322-8156-a0c2c7b408d3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:22.043890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-51507cb3-c214-4322-8156-a0c2c7b408d3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:22.044417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-51507cb3-c214-4322-8156-a0c2c7b408d3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:22.045219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-51507cb3-c214-4322-8156-a0c2c7b408d3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:22.045718 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-51507cb3-c214-4322-8156-a0c2c7b408d3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:22.046186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-51507cb3-c214-4322-8156-a0c2c7b408d3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:22.251505 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-51507cb3-c214-4322-8156-a0c2c7b408d3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:22.265426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-51507cb3-c214-4322-8156-a0c2c7b408d3 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1362 microversion: 2.1 time: 0.314522 Aug 07 01:15:22.266494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 323/668] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:21 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1362 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:23.304058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1962a327-a4d4-430c-9c83-adf702b8f7bb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:23.305577 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1962a327-a4d4-430c-9c83-adf702b8f7bb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:23.316273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1962a327-a4d4-430c-9c83-adf702b8f7bb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:23.316928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1962a327-a4d4-430c-9c83-adf702b8f7bb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:23.317485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1962a327-a4d4-430c-9c83-adf702b8f7bb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:23.318342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1962a327-a4d4-430c-9c83-adf702b8f7bb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:23.319027 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1962a327-a4d4-430c-9c83-adf702b8f7bb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:23.319575 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1962a327-a4d4-430c-9c83-adf702b8f7bb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:23.545214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1962a327-a4d4-430c-9c83-adf702b8f7bb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:23.558357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1962a327-a4d4-430c-9c83-adf702b8f7bb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1362 microversion: 2.1 time: 0.262681 Aug 07 01:15:23.559223 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 346/669] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:23 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1362 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:23.751761 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-9d08cbf9-4fed-487a-a3b9-15f331ca29ed req-dbcc5a59-920b-44b3-93ea-b8dc9f28ea37 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e8c48f1-19d6-4691-9e54-e5d84962485a", "name": "network-vif-plugged", "status": "completed", "tag": "cca96b0c-1343-4c42-8e76-9d5bf2321cee"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:15:23.780139 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-9d08cbf9-4fed-487a-a3b9-15f331ca29ed req-dbcc5a59-920b-44b3-93ea-b8dc9f28ea37 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:23.781244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-9d08cbf9-4fed-487a-a3b9-15f331ca29ed req-dbcc5a59-920b-44b3-93ea-b8dc9f28ea37 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:23.782319 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-9d08cbf9-4fed-487a-a3b9-15f331ca29ed req-dbcc5a59-920b-44b3-93ea-b8dc9f28ea37 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:23.804132 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-9d08cbf9-4fed-487a-a3b9-15f331ca29ed req-dbcc5a59-920b-44b3-93ea-b8dc9f28ea37 service nova] Creating event network-vif-plugged:cca96b0c-1343-4c42-8e76-9d5bf2321cee for instance 8e8c48f1-19d6-4691-9e54-e5d84962485a on hv-cinder-maste Aug 07 01:15:23.814300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-9d08cbf9-4fed-487a-a3b9-15f331ca29ed req-dbcc5a59-920b-44b3-93ea-b8dc9f28ea37 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071507 Aug 07 01:15:23.815409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 324/670] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:15:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:15:24.594739 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c9b619b9-acb4-438f-88c4-835d11fe0b37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:24.595905 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c9b619b9-acb4-438f-88c4-835d11fe0b37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:24.599103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:15:24.606774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c9b619b9-acb4-438f-88c4-835d11fe0b37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:24.607318 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c9b619b9-acb4-438f-88c4-835d11fe0b37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:24.607824 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c9b619b9-acb4-438f-88c4-835d11fe0b37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:24.608682 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c9b619b9-acb4-438f-88c4-835d11fe0b37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:24.609264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c9b619b9-acb4-438f-88c4-835d11fe0b37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:24.609923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c9b619b9-acb4-438f-88c4-835d11fe0b37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:24.610918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=12 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:15:24.823435 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c9b619b9-acb4-438f-88c4-835d11fe0b37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:24.836359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c9b619b9-acb4-438f-88c4-835d11fe0b37 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1362 microversion: 2.1 time: 0.246506 Aug 07 01:15:24.837157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 347/671] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:24 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1362 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:25.871898 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ff7bf461-8fbd-4670-a0d7-d1d5b73cdc9a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:25.873218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ff7bf461-8fbd-4670-a0d7-d1d5b73cdc9a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:25.887332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff7bf461-8fbd-4670-a0d7-d1d5b73cdc9a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:25.888293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff7bf461-8fbd-4670-a0d7-d1d5b73cdc9a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:25.889096 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff7bf461-8fbd-4670-a0d7-d1d5b73cdc9a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:25.890418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff7bf461-8fbd-4670-a0d7-d1d5b73cdc9a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:25.891905 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff7bf461-8fbd-4670-a0d7-d1d5b73cdc9a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:25.892781 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff7bf461-8fbd-4670-a0d7-d1d5b73cdc9a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:26.134523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ff7bf461-8fbd-4670-a0d7-d1d5b73cdc9a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:26.149590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ff7bf461-8fbd-4670-a0d7-d1d5b73cdc9a tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1362 microversion: 2.1 time: 0.284586 Aug 07 01:15:26.150700 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 325/672] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:25 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1362 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:27.179461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d87213e0-000f-40f2-acfb-041710c16a57 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:27.180764 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d87213e0-000f-40f2-acfb-041710c16a57 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:27.194052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d87213e0-000f-40f2-acfb-041710c16a57 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:27.194998 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d87213e0-000f-40f2-acfb-041710c16a57 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:27.195867 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d87213e0-000f-40f2-acfb-041710c16a57 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:27.197426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d87213e0-000f-40f2-acfb-041710c16a57 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:27.198391 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d87213e0-000f-40f2-acfb-041710c16a57 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:27.199452 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d87213e0-000f-40f2-acfb-041710c16a57 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:27.421284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d87213e0-000f-40f2-acfb-041710c16a57 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:27.435431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d87213e0-000f-40f2-acfb-041710c16a57 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1362 microversion: 2.1 time: 0.263630 Aug 07 01:15:27.436047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 348/673] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:27 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1362 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:15:28.460910 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e217a4e2-3226-4b95-96aa-9a0cc6cd6736 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:28.461691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e217a4e2-3226-4b95-96aa-9a0cc6cd6736 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:28.472902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e217a4e2-3226-4b95-96aa-9a0cc6cd6736 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:28.473659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e217a4e2-3226-4b95-96aa-9a0cc6cd6736 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:28.474133 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e217a4e2-3226-4b95-96aa-9a0cc6cd6736 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:28.475694 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e217a4e2-3226-4b95-96aa-9a0cc6cd6736 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:28.476157 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e217a4e2-3226-4b95-96aa-9a0cc6cd6736 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:28.476789 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e217a4e2-3226-4b95-96aa-9a0cc6cd6736 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:28.674435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e217a4e2-3226-4b95-96aa-9a0cc6cd6736 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:28.685361 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e217a4e2-3226-4b95-96aa-9a0cc6cd6736 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1362 microversion: 2.1 time: 0.229807 Aug 07 01:15:28.686113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 326/674] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:28 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1362 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:29.709706 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8a813856-c1c7-49c1-bede-7374174aa2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:29.710405 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8a813856-c1c7-49c1-bede-7374174aa2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:29.718665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a813856-c1c7-49c1-bede-7374174aa2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:29.719960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a813856-c1c7-49c1-bede-7374174aa2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:29.720529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a813856-c1c7-49c1-bede-7374174aa2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:29.721384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a813856-c1c7-49c1-bede-7374174aa2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:29.721949 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a813856-c1c7-49c1-bede-7374174aa2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:29.722443 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a813856-c1c7-49c1-bede-7374174aa2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:29.726070 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:15:29.935075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8a813856-c1c7-49c1-bede-7374174aa2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:15:29.954463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8a813856-c1c7-49c1-bede-7374174aa2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1548 microversion: 2.1 time: 0.248005 Aug 07 01:15:29.968927 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 349/675] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:15:29 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1548 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:15:29.995067 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2f45bbc4-b005-4b8a-846f-605188cea5f2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:29.996042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2f45bbc4-b005-4b8a-846f-605188cea5f2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:30.009750 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f45bbc4-b005-4b8a-846f-605188cea5f2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:30.010609 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f45bbc4-b005-4b8a-846f-605188cea5f2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:30.011398 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f45bbc4-b005-4b8a-846f-605188cea5f2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:30.012730 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f45bbc4-b005-4b8a-846f-605188cea5f2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:30.013632 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f45bbc4-b005-4b8a-846f-605188cea5f2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:30.014557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f45bbc4-b005-4b8a-846f-605188cea5f2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:30.143118 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2f45bbc4-b005-4b8a-846f-605188cea5f2 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.160329 Aug 07 01:15:30.143807 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 327/676] 10.222.0.26 () {58 vars in 1275 bytes} [Sun Aug 7 01:15:29 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a/os-interface => generated 284 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:15:31.829120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-e05e8e9c-69a4-4918-a8b7-d1a8bc9e2f36 req-60dc9889-27ef-4416-bdae-5efb814e99c3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8e8c48f1-19d6-4691-9e54-e5d84962485a", "tag": "cca96b0c-1343-4c42-8e76-9d5bf2321cee"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:15:31.849628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-e05e8e9c-69a4-4918-a8b7-d1a8bc9e2f36 req-60dc9889-27ef-4416-bdae-5efb814e99c3 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:31.850523 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-e05e8e9c-69a4-4918-a8b7-d1a8bc9e2f36 req-60dc9889-27ef-4416-bdae-5efb814e99c3 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:31.851316 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-e05e8e9c-69a4-4918-a8b7-d1a8bc9e2f36 req-60dc9889-27ef-4416-bdae-5efb814e99c3 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:31.873697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-e05e8e9c-69a4-4918-a8b7-d1a8bc9e2f36 req-60dc9889-27ef-4416-bdae-5efb814e99c3 service nova] Creating event network-changed:cca96b0c-1343-4c42-8e76-9d5bf2321cee for instance 8e8c48f1-19d6-4691-9e54-e5d84962485a on hv-cinder-maste Aug 07 01:15:31.882975 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-e05e8e9c-69a4-4918-a8b7-d1a8bc9e2f36 req-60dc9889-27ef-4416-bdae-5efb814e99c3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061021 Aug 07 01:15:31.884194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 350/677] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:15:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:15:32.690037 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e33483ca-c75d-42b1-9b61-e361c4feaf84 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:32.690942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e33483ca-c75d-42b1-9b61-e361c4feaf84 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:32.702056 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e33483ca-c75d-42b1-9b61-e361c4feaf84 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:32.703074 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e33483ca-c75d-42b1-9b61-e361c4feaf84 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:32.704433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e33483ca-c75d-42b1-9b61-e361c4feaf84 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:32.705885 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e33483ca-c75d-42b1-9b61-e361c4feaf84 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:32.706625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e33483ca-c75d-42b1-9b61-e361c4feaf84 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:32.707376 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e33483ca-c75d-42b1-9b61-e361c4feaf84 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:32.756109 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e33483ca-c75d-42b1-9b61-e361c4feaf84 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222/ips" status: 200 len: 151 microversion: 2.1 time: 0.164081 Aug 07 01:15:32.757479 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 328/678] 10.222.0.26 () {58 vars in 1248 bytes} [Sun Aug 7 01:15:32 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222/ips => generated 151 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:15:55.205714 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=58,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:15:55.206675 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=7 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:15:55.213352 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c30ba53e-7b27-490f-b63c-1445a97e9293 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:15:55.214114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c30ba53e-7b27-490f-b63c-1445a97e9293 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:15:55.227693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c30ba53e-7b27-490f-b63c-1445a97e9293 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:55.228633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c30ba53e-7b27-490f-b63c-1445a97e9293 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:55.229434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c30ba53e-7b27-490f-b63c-1445a97e9293 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:55.230762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c30ba53e-7b27-490f-b63c-1445a97e9293 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:15:55.231626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c30ba53e-7b27-490f-b63c-1445a97e9293 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:15:55.232459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c30ba53e-7b27-490f-b63c-1445a97e9293 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:15:55.273488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c30ba53e-7b27-490f-b63c-1445a97e9293 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a/ips" status: 200 len: 153 microversion: 2.20 time: 0.069770 Aug 07 01:15:55.274586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 351/679] 10.222.0.26 () {60 vars in 1289 bytes} [Sun Aug 7 01:15:55 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a/ips => generated 153 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 07 01:16:08.690156 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-223dda78-ad03-4b3e-8412-3312642c9038 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:08.691087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-223dda78-ad03-4b3e-8412-3312642c9038 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:08.695294 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=25 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:16:08.696498 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=77 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:16:08.708547 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-223dda78-ad03-4b3e-8412-3312642c9038 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:08.709436 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-223dda78-ad03-4b3e-8412-3312642c9038 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:08.710220 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-223dda78-ad03-4b3e-8412-3312642c9038 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:08.711535 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-223dda78-ad03-4b3e-8412-3312642c9038 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:08.712444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-223dda78-ad03-4b3e-8412-3312642c9038 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:08.713221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-223dda78-ad03-4b3e-8412-3312642c9038 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:08.766618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-223dda78-ad03-4b3e-8412-3312642c9038 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a/ips" status: 200 len: 106 microversion: 2.1 time: 0.083757 Aug 07 01:16:08.767773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 329/680] 10.222.0.26 () {58 vars in 1248 bytes} [Sun Aug 7 01:16:08 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a/ips => generated 106 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:16:19.595004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-201b8b3c-2180-4617-94ee-0774b1381988 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:16:19.599314 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-201b8b3c-2180-4617-94ee-0774b1381988 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:19.603647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:16:19.605976 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=4 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:16:19.618395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-201b8b3c-2180-4617-94ee-0774b1381988 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:19.619860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-201b8b3c-2180-4617-94ee-0774b1381988 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:19.621108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-201b8b3c-2180-4617-94ee-0774b1381988 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:19.622905 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-201b8b3c-2180-4617-94ee-0774b1381988 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:19.624082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-201b8b3c-2180-4617-94ee-0774b1381988 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:19.625672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-201b8b3c-2180-4617-94ee-0774b1381988 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:19.682380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.objects.instance [None req-201b8b3c-2180-4617-94ee-0774b1381988 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lazy-loading 'flavor' on Instance uuid 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd {{(pid=117983) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:16:19.988355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-201b8b3c-2180-4617-94ee-0774b1381988 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "POST /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/action" status: 200 len: 32 microversion: 2.1 time: 0.398847 Aug 07 01:16:19.990201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 352/681] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:16:19 2022] POST /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/action => generated 32 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 07 01:16:20.010097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1216f039-3290-4f2b-8859-142d7aab9a19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:20.012835 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1216f039-3290-4f2b-8859-142d7aab9a19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:20.021459 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:16:20.023463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=4 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:16:20.034480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1216f039-3290-4f2b-8859-142d7aab9a19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:20.035768 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1216f039-3290-4f2b-8859-142d7aab9a19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:20.036846 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1216f039-3290-4f2b-8859-142d7aab9a19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:20.038058 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1216f039-3290-4f2b-8859-142d7aab9a19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:20.038910 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1216f039-3290-4f2b-8859-142d7aab9a19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:20.039952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1216f039-3290-4f2b-8859-142d7aab9a19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:20.323587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1216f039-3290-4f2b-8859-142d7aab9a19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:20.353309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1216f039-3290-4f2b-8859-142d7aab9a19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1736 microversion: 2.1 time: 0.344755 Aug 07 01:16:20.353309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 330/682] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:19 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1736 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:21.390191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-eb94ff65-b182-43ad-a414-35681626588d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:21.392042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-eb94ff65-b182-43ad-a414-35681626588d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:21.404782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb94ff65-b182-43ad-a414-35681626588d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:21.405566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb94ff65-b182-43ad-a414-35681626588d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:21.406854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb94ff65-b182-43ad-a414-35681626588d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:21.408196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb94ff65-b182-43ad-a414-35681626588d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:21.409087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb94ff65-b182-43ad-a414-35681626588d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:21.409849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb94ff65-b182-43ad-a414-35681626588d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:21.611071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-eb94ff65-b182-43ad-a414-35681626588d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:21.620856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-eb94ff65-b182-43ad-a414-35681626588d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1736 microversion: 2.1 time: 0.240658 Aug 07 01:16:21.621529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 353/683] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:21 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1736 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:22.648762 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ede4d02b-7ff2-4b02-bb72-90e0595815ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:22.650067 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ede4d02b-7ff2-4b02-bb72-90e0595815ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:22.664603 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ede4d02b-7ff2-4b02-bb72-90e0595815ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:22.665508 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ede4d02b-7ff2-4b02-bb72-90e0595815ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:22.666312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ede4d02b-7ff2-4b02-bb72-90e0595815ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:22.667823 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ede4d02b-7ff2-4b02-bb72-90e0595815ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:22.668794 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ede4d02b-7ff2-4b02-bb72-90e0595815ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:22.669497 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ede4d02b-7ff2-4b02-bb72-90e0595815ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:22.950234 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ede4d02b-7ff2-4b02-bb72-90e0595815ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:22.967644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ede4d02b-7ff2-4b02-bb72-90e0595815ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1736 microversion: 2.1 time: 0.326570 Aug 07 01:16:22.968764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 331/684] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:22 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1736 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:24.003554 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-34e60552-b17d-4668-9ba9-dccfbaa66d9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:24.004467 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-34e60552-b17d-4668-9ba9-dccfbaa66d9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:24.016545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-34e60552-b17d-4668-9ba9-dccfbaa66d9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:24.017427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-34e60552-b17d-4668-9ba9-dccfbaa66d9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:24.018332 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-34e60552-b17d-4668-9ba9-dccfbaa66d9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:24.019571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-34e60552-b17d-4668-9ba9-dccfbaa66d9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:24.020398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-34e60552-b17d-4668-9ba9-dccfbaa66d9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:24.021136 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-34e60552-b17d-4668-9ba9-dccfbaa66d9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:24.279104 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-34e60552-b17d-4668-9ba9-dccfbaa66d9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:24.294980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-34e60552-b17d-4668-9ba9-dccfbaa66d9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1736 microversion: 2.1 time: 0.298326 Aug 07 01:16:24.296046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 354/685] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:23 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1736 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:25.339121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0d622de8-c64a-4545-8250-27ae3e1a29a9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:25.340432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0d622de8-c64a-4545-8250-27ae3e1a29a9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:25.353591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d622de8-c64a-4545-8250-27ae3e1a29a9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:25.354499 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d622de8-c64a-4545-8250-27ae3e1a29a9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:25.355297 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d622de8-c64a-4545-8250-27ae3e1a29a9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:25.356705 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d622de8-c64a-4545-8250-27ae3e1a29a9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:25.357582 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d622de8-c64a-4545-8250-27ae3e1a29a9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:25.358370 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d622de8-c64a-4545-8250-27ae3e1a29a9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:25.647592 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0d622de8-c64a-4545-8250-27ae3e1a29a9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:25.663813 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0d622de8-c64a-4545-8250-27ae3e1a29a9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1736 microversion: 2.1 time: 0.332186 Aug 07 01:16:25.664936 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 332/686] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:25 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1736 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:26.704779 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7be9074e-eb92-4d33-abae-5182f1e4dfd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:26.705970 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7be9074e-eb92-4d33-abae-5182f1e4dfd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:26.718423 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7be9074e-eb92-4d33-abae-5182f1e4dfd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:26.719804 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7be9074e-eb92-4d33-abae-5182f1e4dfd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:26.720562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7be9074e-eb92-4d33-abae-5182f1e4dfd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:26.721930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7be9074e-eb92-4d33-abae-5182f1e4dfd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:26.722765 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7be9074e-eb92-4d33-abae-5182f1e4dfd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:26.723514 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7be9074e-eb92-4d33-abae-5182f1e4dfd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:26.927729 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7be9074e-eb92-4d33-abae-5182f1e4dfd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:26.943374 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7be9074e-eb92-4d33-abae-5182f1e4dfd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1736 microversion: 2.1 time: 0.246085 Aug 07 01:16:26.944516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 355/687] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:26 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1736 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:27.987046 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-977b819c-6c54-4ed4-858e-a6af6e2c4338 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:27.988171 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-977b819c-6c54-4ed4-858e-a6af6e2c4338 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:28.001341 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-977b819c-6c54-4ed4-858e-a6af6e2c4338 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:28.002266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-977b819c-6c54-4ed4-858e-a6af6e2c4338 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:28.002968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-977b819c-6c54-4ed4-858e-a6af6e2c4338 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:28.004391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-977b819c-6c54-4ed4-858e-a6af6e2c4338 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:28.005623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-977b819c-6c54-4ed4-858e-a6af6e2c4338 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:28.006686 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-977b819c-6c54-4ed4-858e-a6af6e2c4338 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:28.285735 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-977b819c-6c54-4ed4-858e-a6af6e2c4338 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:28.298729 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-977b819c-6c54-4ed4-858e-a6af6e2c4338 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1736 microversion: 2.1 time: 0.318948 Aug 07 01:16:28.299337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 333/688] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:27 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1736 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:29.334649 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-224c1def-3d7c-4ea7-916f-5857e44315cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:29.335568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-224c1def-3d7c-4ea7-916f-5857e44315cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:29.345487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-224c1def-3d7c-4ea7-916f-5857e44315cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:29.346093 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-224c1def-3d7c-4ea7-916f-5857e44315cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:29.346671 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-224c1def-3d7c-4ea7-916f-5857e44315cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:29.347542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-224c1def-3d7c-4ea7-916f-5857e44315cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:29.348023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-224c1def-3d7c-4ea7-916f-5857e44315cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:29.348552 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-224c1def-3d7c-4ea7-916f-5857e44315cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:29.532368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-224c1def-3d7c-4ea7-916f-5857e44315cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:29.549563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-224c1def-3d7c-4ea7-916f-5857e44315cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1736 microversion: 2.1 time: 0.220790 Aug 07 01:16:29.550309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 356/689] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:29 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1736 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:30.578457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f09b752b-4804-4d7a-82c7-ecc8144e66b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:30.579489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f09b752b-4804-4d7a-82c7-ecc8144e66b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:30.590769 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f09b752b-4804-4d7a-82c7-ecc8144e66b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:30.591597 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f09b752b-4804-4d7a-82c7-ecc8144e66b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:30.592392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f09b752b-4804-4d7a-82c7-ecc8144e66b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:30.593766 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f09b752b-4804-4d7a-82c7-ecc8144e66b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:30.595092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f09b752b-4804-4d7a-82c7-ecc8144e66b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:30.595760 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f09b752b-4804-4d7a-82c7-ecc8144e66b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:30.847437 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f09b752b-4804-4d7a-82c7-ecc8144e66b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:30.861874 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f09b752b-4804-4d7a-82c7-ecc8144e66b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1736 microversion: 2.1 time: 0.291353 Aug 07 01:16:30.862975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 334/690] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:30 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1736 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:31.893735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6bdd4a37-08d6-48ee-986f-3ff879fbd6bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:31.895792 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6bdd4a37-08d6-48ee-986f-3ff879fbd6bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:31.907407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6bdd4a37-08d6-48ee-986f-3ff879fbd6bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:31.908278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6bdd4a37-08d6-48ee-986f-3ff879fbd6bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:31.909030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6bdd4a37-08d6-48ee-986f-3ff879fbd6bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:31.910206 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6bdd4a37-08d6-48ee-986f-3ff879fbd6bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:31.911011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6bdd4a37-08d6-48ee-986f-3ff879fbd6bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:31.911692 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6bdd4a37-08d6-48ee-986f-3ff879fbd6bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:32.127372 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6bdd4a37-08d6-48ee-986f-3ff879fbd6bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:32.140471 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6bdd4a37-08d6-48ee-986f-3ff879fbd6bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1736 microversion: 2.1 time: 0.253651 Aug 07 01:16:32.141932 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 357/691] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:31 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1736 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:33.174028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1d03ecf2-48b7-488a-aabd-07823c02d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:33.176955 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1d03ecf2-48b7-488a-aabd-07823c02d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:33.189689 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d03ecf2-48b7-488a-aabd-07823c02d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:33.190558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d03ecf2-48b7-488a-aabd-07823c02d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:33.191108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d03ecf2-48b7-488a-aabd-07823c02d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:33.191966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d03ecf2-48b7-488a-aabd-07823c02d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:33.192548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d03ecf2-48b7-488a-aabd-07823c02d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:33.193226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d03ecf2-48b7-488a-aabd-07823c02d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:33.452342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1d03ecf2-48b7-488a-aabd-07823c02d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:33.466610 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1d03ecf2-48b7-488a-aabd-07823c02d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1736 microversion: 2.1 time: 0.304396 Aug 07 01:16:33.467666 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 335/692] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:33 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1736 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:34.495766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ac5d38f9-5214-4b1d-8f04-b75726bfa2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:34.496732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ac5d38f9-5214-4b1d-8f04-b75726bfa2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:34.509697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac5d38f9-5214-4b1d-8f04-b75726bfa2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:34.510616 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac5d38f9-5214-4b1d-8f04-b75726bfa2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:34.511419 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac5d38f9-5214-4b1d-8f04-b75726bfa2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:34.512805 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac5d38f9-5214-4b1d-8f04-b75726bfa2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:34.513661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac5d38f9-5214-4b1d-8f04-b75726bfa2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:34.514448 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac5d38f9-5214-4b1d-8f04-b75726bfa2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:34.742462 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ac5d38f9-5214-4b1d-8f04-b75726bfa2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:34.758841 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ac5d38f9-5214-4b1d-8f04-b75726bfa2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1736 microversion: 2.1 time: 0.269896 Aug 07 01:16:34.759817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 358/693] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:34 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1736 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:35.801734 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b64f9a09-36ff-47ea-9cdd-88aa4bd2771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:35.802728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b64f9a09-36ff-47ea-9cdd-88aa4bd2771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:35.820517 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b64f9a09-36ff-47ea-9cdd-88aa4bd2771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:35.821349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b64f9a09-36ff-47ea-9cdd-88aa4bd2771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:35.822040 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b64f9a09-36ff-47ea-9cdd-88aa4bd2771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:35.823253 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b64f9a09-36ff-47ea-9cdd-88aa4bd2771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:35.824080 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b64f9a09-36ff-47ea-9cdd-88aa4bd2771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:35.824885 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b64f9a09-36ff-47ea-9cdd-88aa4bd2771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:36.084944 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b64f9a09-36ff-47ea-9cdd-88aa4bd2771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:36.101954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b64f9a09-36ff-47ea-9cdd-88aa4bd2771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1716 microversion: 2.1 time: 0.307923 Aug 07 01:16:36.103121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 336/694] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:35 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1716 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:36.210071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-948f0bb8-06bf-4c8f-883d-08552a4eaee5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:36.210877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-948f0bb8-06bf-4c8f-883d-08552a4eaee5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:36.218505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-948f0bb8-06bf-4c8f-883d-08552a4eaee5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:36.219594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-948f0bb8-06bf-4c8f-883d-08552a4eaee5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:36.220092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-948f0bb8-06bf-4c8f-883d-08552a4eaee5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:36.220939 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-948f0bb8-06bf-4c8f-883d-08552a4eaee5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:36.221470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-948f0bb8-06bf-4c8f-883d-08552a4eaee5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:36.221941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-948f0bb8-06bf-4c8f-883d-08552a4eaee5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:36.387892 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-948f0bb8-06bf-4c8f-883d-08552a4eaee5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] HTTP exception thrown: Cannot 'detach_volume' instance 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd while it is in vm_state rescued Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-948f0bb8-06bf-4c8f-883d-08552a4eaee5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Request method failure captured: Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: DELETE /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments/49c52ecb-fd7f-409d-b4c7-6f5bb414da0f HTTP/1.1 Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xFan_-oNUM4AqSt7Wni9p606OoWcdJLz4SIr5VHLwgODJGL0BBSKB64xI7IBMYJhEzGuAdjPCyw6B3DmJ9opPnSTNGpjpiLZ8MHcy7T0m2nWHFImY2c23r73oTOGBUNFDYYRljAnrO4awdzCByqv2_7MGWKYhYTsQ-EtnOqUmEQZ_Q Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 9badaa18dcf64810b10dd7bf03d8e40d Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1786214574 Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:16:36.392412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:16:36.395198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:16:36.395198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1786214574 Aug 07 01:16:36.395198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 9badaa18dcf64810b10dd7bf03d8e40d Aug 07 01:16:36.395198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1786214574 Aug 07 01:16:36.395198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-ServerRescueNegativeTestJSON-1786214574-project Aug 07 01:16:36.395198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:16:36.395198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:16:36.395198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: ec931c9cc56a462991413926d34dfa46 Aug 07 01:16:36.395198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1786214574-project Aug 07 01:16:36.395198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:16:36.395198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Cannot 'detach_volume' instance 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd while it is in vm_state rescued Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'server_id': '6ac28640-d6da-4cfc-b7ad-0d69136ac1fd', 'id': '49c52ecb-fd7f-409d-b4c7-6f5bb414da0f'} Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd in vm_state rescued. Cannot detach_volume while the instance is in this state. Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:16:36.396608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd while it is in vm_state rescued Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd while it is in vm_state rescued Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:16:36.399010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-948f0bb8-06bf-4c8f-883d-08552a4eaee5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Returning 409 to user: Cannot 'detach_volume' instance 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd while it is in vm_state rescued {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:16:36.401724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-948f0bb8-06bf-4c8f-883d-08552a4eaee5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "DELETE /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments/49c52ecb-fd7f-409d-b4c7-6f5bb414da0f" status: 409 len: 152 microversion: 2.1 time: 0.190319 Aug 07 01:16:36.401724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 359/695] 10.222.0.26 () {58 vars in 1416 bytes} [Sun Aug 7 01:16:36 2022] DELETE /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments/49c52ecb-fd7f-409d-b4c7-6f5bb414da0f => generated 152 bytes in 192 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Aug 07 01:16:36.413145 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1b5b004a-9ead-4cb4-b25a-4b85b15647c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:16:36.414237 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1b5b004a-9ead-4cb4-b25a-4b85b15647c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:36.424927 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5b004a-9ead-4cb4-b25a-4b85b15647c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:36.425851 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5b004a-9ead-4cb4-b25a-4b85b15647c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:36.426875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5b004a-9ead-4cb4-b25a-4b85b15647c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:36.427942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5b004a-9ead-4cb4-b25a-4b85b15647c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:36.428873 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5b004a-9ead-4cb4-b25a-4b85b15647c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:36.429702 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5b004a-9ead-4cb4-b25a-4b85b15647c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:36.566248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1b5b004a-9ead-4cb4-b25a-4b85b15647c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "POST /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/action" status: 202 len: 0 microversion: 2.1 time: 0.160415 Aug 07 01:16:36.567657 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 337/696] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:16:36 2022] POST /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/action => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:16:36.581530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-19bd726a-14f1-407f-8ce2-1b288490c278 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:36.582773 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-19bd726a-14f1-407f-8ce2-1b288490c278 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:36.594696 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19bd726a-14f1-407f-8ce2-1b288490c278 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:36.595747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19bd726a-14f1-407f-8ce2-1b288490c278 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:36.596647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19bd726a-14f1-407f-8ce2-1b288490c278 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:36.597681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19bd726a-14f1-407f-8ce2-1b288490c278 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:36.598782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19bd726a-14f1-407f-8ce2-1b288490c278 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:36.599683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19bd726a-14f1-407f-8ce2-1b288490c278 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:37.052821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-19bd726a-14f1-407f-8ce2-1b288490c278 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:37.070166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-19bd726a-14f1-407f-8ce2-1b288490c278 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1724 microversion: 2.1 time: 0.496137 Aug 07 01:16:37.071723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 360/697] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:36 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1724 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:16:38.112783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-242ce917-a312-4e01-a0d9-cd5ca85da0df tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:38.114277 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-242ce917-a312-4e01-a0d9-cd5ca85da0df tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:38.132439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-242ce917-a312-4e01-a0d9-cd5ca85da0df tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:38.132439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-242ce917-a312-4e01-a0d9-cd5ca85da0df tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:38.132439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-242ce917-a312-4e01-a0d9-cd5ca85da0df tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:38.132439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-242ce917-a312-4e01-a0d9-cd5ca85da0df tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:38.132930 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-242ce917-a312-4e01-a0d9-cd5ca85da0df tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:38.132930 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-242ce917-a312-4e01-a0d9-cd5ca85da0df tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:38.320389 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-242ce917-a312-4e01-a0d9-cd5ca85da0df tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:38.331213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-242ce917-a312-4e01-a0d9-cd5ca85da0df tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1724 microversion: 2.1 time: 0.227108 Aug 07 01:16:38.331986 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 338/698] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:38 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1724 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:39.360738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3bdd3d35-8078-4f31-b8e9-e08408170d59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:39.361895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3bdd3d35-8078-4f31-b8e9-e08408170d59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:39.373932 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3bdd3d35-8078-4f31-b8e9-e08408170d59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:39.375589 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3bdd3d35-8078-4f31-b8e9-e08408170d59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:39.376421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3bdd3d35-8078-4f31-b8e9-e08408170d59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:39.377731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3bdd3d35-8078-4f31-b8e9-e08408170d59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:39.378547 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3bdd3d35-8078-4f31-b8e9-e08408170d59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:39.379334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3bdd3d35-8078-4f31-b8e9-e08408170d59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:39.587645 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3bdd3d35-8078-4f31-b8e9-e08408170d59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:39.605166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3bdd3d35-8078-4f31-b8e9-e08408170d59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1724 microversion: 2.1 time: 0.252217 Aug 07 01:16:39.608294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 361/699] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:39 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1724 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:16:40.632932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-60a4fcbd-6694-4bbe-8e17-59762b2da485 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:40.633809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-60a4fcbd-6694-4bbe-8e17-59762b2da485 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:40.643990 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60a4fcbd-6694-4bbe-8e17-59762b2da485 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:40.644574 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60a4fcbd-6694-4bbe-8e17-59762b2da485 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:40.645126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60a4fcbd-6694-4bbe-8e17-59762b2da485 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:40.645933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60a4fcbd-6694-4bbe-8e17-59762b2da485 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:40.646454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60a4fcbd-6694-4bbe-8e17-59762b2da485 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:40.647015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60a4fcbd-6694-4bbe-8e17-59762b2da485 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:40.898097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-60a4fcbd-6694-4bbe-8e17-59762b2da485 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:40.914319 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-60a4fcbd-6694-4bbe-8e17-59762b2da485 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1730 microversion: 2.1 time: 0.285655 Aug 07 01:16:40.915406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 339/700] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:40 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1730 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:16:40.956262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b0ca3af7-89ce-4cdb-8aac-4b7e6357120c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:40.957165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b0ca3af7-89ce-4cdb-8aac-4b7e6357120c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:40.969593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b0ca3af7-89ce-4cdb-8aac-4b7e6357120c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:40.970489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b0ca3af7-89ce-4cdb-8aac-4b7e6357120c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:40.971297 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b0ca3af7-89ce-4cdb-8aac-4b7e6357120c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:40.972648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b0ca3af7-89ce-4cdb-8aac-4b7e6357120c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:40.973497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b0ca3af7-89ce-4cdb-8aac-4b7e6357120c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:40.974247 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b0ca3af7-89ce-4cdb-8aac-4b7e6357120c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:41.122842 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b0ca3af7-89ce-4cdb-8aac-4b7e6357120c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.174739 Aug 07 01:16:41.123902 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 362/701] 10.222.0.26 () {58 vars in 1275 bytes} [Sun Aug 7 01:16:40 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-interface => generated 285 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:16:43.098681 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-2277665c-0ae0-48d4-b825-71892d548f03 req-4b649c47-aa72-4bb2-b1e5-273a6e60a061 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6ac28640-d6da-4cfc-b7ad-0d69136ac1fd", "tag": "4d7d0711-c6ef-4ae2-b272-c2cf175993d4"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:16:43.110599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-2277665c-0ae0-48d4-b825-71892d548f03 req-4b649c47-aa72-4bb2-b1e5-273a6e60a061 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:43.111148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-2277665c-0ae0-48d4-b825-71892d548f03 req-4b649c47-aa72-4bb2-b1e5-273a6e60a061 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:43.111685 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-2277665c-0ae0-48d4-b825-71892d548f03 req-4b649c47-aa72-4bb2-b1e5-273a6e60a061 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:43.126408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-2277665c-0ae0-48d4-b825-71892d548f03 req-4b649c47-aa72-4bb2-b1e5-273a6e60a061 service nova] Creating event network-changed:4d7d0711-c6ef-4ae2-b272-c2cf175993d4 for instance 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd on hv-cinder-maste Aug 07 01:16:43.135836 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-2277665c-0ae0-48d4-b825-71892d548f03 req-4b649c47-aa72-4bb2-b1e5-273a6e60a061 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.183738 Aug 07 01:16:43.136960 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 340/702] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:16:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:16:45.161656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-a6a452db-deee-4153-8a20-eb0c1c024741 req-42698b8e-59ea-4857-948d-7e12ae5dc6ea service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6ac28640-d6da-4cfc-b7ad-0d69136ac1fd", "tag": "4d7d0711-c6ef-4ae2-b272-c2cf175993d4"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:16:45.179698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a6a452db-deee-4153-8a20-eb0c1c024741 req-42698b8e-59ea-4857-948d-7e12ae5dc6ea service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:45.180538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a6a452db-deee-4153-8a20-eb0c1c024741 req-42698b8e-59ea-4857-948d-7e12ae5dc6ea service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:45.181335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-a6a452db-deee-4153-8a20-eb0c1c024741 req-42698b8e-59ea-4857-948d-7e12ae5dc6ea service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:45.205174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-a6a452db-deee-4153-8a20-eb0c1c024741 req-42698b8e-59ea-4857-948d-7e12ae5dc6ea service nova] Creating event network-changed:4d7d0711-c6ef-4ae2-b272-c2cf175993d4 for instance 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd on hv-cinder-maste Aug 07 01:16:45.215152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-a6a452db-deee-4153-8a20-eb0c1c024741 req-42698b8e-59ea-4857-948d-7e12ae5dc6ea service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060815 Aug 07 01:16:45.216514 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 363/703] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:16:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:16:58.647513 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=22 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:16:58.649072 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=66,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:16:58.658156 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-51ce8f5e-8a75-4b2e-a4c2-b505a36a6911 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:58.659256 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-51ce8f5e-8a75-4b2e-a4c2-b505a36a6911 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:58.673760 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-51ce8f5e-8a75-4b2e-a4c2-b505a36a6911 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:58.677991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-51ce8f5e-8a75-4b2e-a4c2-b505a36a6911 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:58.677991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-51ce8f5e-8a75-4b2e-a4c2-b505a36a6911 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:58.679367 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-51ce8f5e-8a75-4b2e-a4c2-b505a36a6911 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:58.680269 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-51ce8f5e-8a75-4b2e-a4c2-b505a36a6911 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:58.681051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-51ce8f5e-8a75-4b2e-a4c2-b505a36a6911 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:58.740746 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-51ce8f5e-8a75-4b2e-a4c2-b505a36a6911 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Going to try to terminate instance {{(pid=117984) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:16:58.865090 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-51ce8f5e-8a75-4b2e-a4c2-b505a36a6911 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "DELETE /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 204 len: 0 microversion: 2.1 time: 0.220175 Aug 07 01:16:58.865584 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 341/704] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:16:58 2022] DELETE /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 07 01:16:58.872875 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=24 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:16:58.874028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=75,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:16:58.879951 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b3f96bf0-4ca8-493b-b97b-84b42c7071a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:16:58.881448 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b3f96bf0-4ca8-493b-b97b-84b42c7071a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:16:58.892135 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b3f96bf0-4ca8-493b-b97b-84b42c7071a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:58.892877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b3f96bf0-4ca8-493b-b97b-84b42c7071a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:58.893367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b3f96bf0-4ca8-493b-b97b-84b42c7071a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:58.894272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b3f96bf0-4ca8-493b-b97b-84b42c7071a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:16:58.894823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b3f96bf0-4ca8-493b-b97b-84b42c7071a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:16:58.895290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b3f96bf0-4ca8-493b-b97b-84b42c7071a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:16:59.065739 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b3f96bf0-4ca8-493b-b97b-84b42c7071a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:16:59.077459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b3f96bf0-4ca8-493b-b97b-84b42c7071a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1577 microversion: 2.1 time: 0.206397 Aug 07 01:16:59.078663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 364/705] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:16:58 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1577 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:00.103939 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-aff468a5-a76e-41f3-8902-3a7d599a7318 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:00.104971 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-aff468a5-a76e-41f3-8902-3a7d599a7318 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:00.113105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aff468a5-a76e-41f3-8902-3a7d599a7318 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:00.114000 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aff468a5-a76e-41f3-8902-3a7d599a7318 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:00.114821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aff468a5-a76e-41f3-8902-3a7d599a7318 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:00.116197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aff468a5-a76e-41f3-8902-3a7d599a7318 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:00.117136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aff468a5-a76e-41f3-8902-3a7d599a7318 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:00.117932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aff468a5-a76e-41f3-8902-3a7d599a7318 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:00.338404 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-aff468a5-a76e-41f3-8902-3a7d599a7318 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:00.349287 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-aff468a5-a76e-41f3-8902-3a7d599a7318 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1577 microversion: 2.1 time: 0.250437 Aug 07 01:17:00.349999 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 342/706] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:00 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1577 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:01.374290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f963f71c-86aa-47c4-925a-8d95bd983a57 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:01.375794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f963f71c-86aa-47c4-925a-8d95bd983a57 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:01.384256 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f963f71c-86aa-47c4-925a-8d95bd983a57 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:01.384776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f963f71c-86aa-47c4-925a-8d95bd983a57 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:01.385218 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f963f71c-86aa-47c4-925a-8d95bd983a57 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:01.385955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f963f71c-86aa-47c4-925a-8d95bd983a57 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:01.386558 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f963f71c-86aa-47c4-925a-8d95bd983a57 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:01.387343 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f963f71c-86aa-47c4-925a-8d95bd983a57 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:01.705257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f963f71c-86aa-47c4-925a-8d95bd983a57 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:01.719029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f963f71c-86aa-47c4-925a-8d95bd983a57 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1577 microversion: 2.1 time: 0.349842 Aug 07 01:17:01.720001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 365/707] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:01 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1577 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:02.543155 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-9909d056-90c8-4a29-a6e3-2fbee23155e1 req-fbe4a1a0-cb3d-4188-8198-bf5a5920092a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9968bbaa-9908-4063-bc7f-e6e805385fa2", "name": "network-vif-deleted", "tag": "6681d8c9-7955-47db-a880-4ab2cb9ac6d1"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:17:02.556213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-9909d056-90c8-4a29-a6e3-2fbee23155e1 req-fbe4a1a0-cb3d-4188-8198-bf5a5920092a service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:02.556810 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-9909d056-90c8-4a29-a6e3-2fbee23155e1 req-fbe4a1a0-cb3d-4188-8198-bf5a5920092a service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:02.557320 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-9909d056-90c8-4a29-a6e3-2fbee23155e1 req-fbe4a1a0-cb3d-4188-8198-bf5a5920092a service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:02.583634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-9909d056-90c8-4a29-a6e3-2fbee23155e1 req-fbe4a1a0-cb3d-4188-8198-bf5a5920092a service nova] Creating event network-vif-deleted:6681d8c9-7955-47db-a880-4ab2cb9ac6d1 for instance 9968bbaa-9908-4063-bc7f-e6e805385fa2 on hv-cinder-maste Aug 07 01:17:02.591842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-9909d056-90c8-4a29-a6e3-2fbee23155e1 req-fbe4a1a0-cb3d-4188-8198-bf5a5920092a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.142674 Aug 07 01:17:02.593176 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 343/708] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:17:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:02.758720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-77057207-fb36-48a0-85a5-c73a9e8175c0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:02.759782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-77057207-fb36-48a0-85a5-c73a9e8175c0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:02.768175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77057207-fb36-48a0-85a5-c73a9e8175c0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:02.769063 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77057207-fb36-48a0-85a5-c73a9e8175c0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:02.769775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77057207-fb36-48a0-85a5-c73a9e8175c0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:02.770803 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77057207-fb36-48a0-85a5-c73a9e8175c0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:02.771457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77057207-fb36-48a0-85a5-c73a9e8175c0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:02.772109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77057207-fb36-48a0-85a5-c73a9e8175c0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:02.877053 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-77057207-fb36-48a0-85a5-c73a9e8175c0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:02.886379 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-77057207-fb36-48a0-85a5-c73a9e8175c0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.133139 Aug 07 01:17:02.887158 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 366/709] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:02 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:03.925090 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7cb22259-7d64-4299-b4f8-effbe2ec1829 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:03.926356 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7cb22259-7d64-4299-b4f8-effbe2ec1829 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:03.937027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7cb22259-7d64-4299-b4f8-effbe2ec1829 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:03.937708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7cb22259-7d64-4299-b4f8-effbe2ec1829 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:03.938313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7cb22259-7d64-4299-b4f8-effbe2ec1829 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:03.939217 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7cb22259-7d64-4299-b4f8-effbe2ec1829 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:03.939755 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7cb22259-7d64-4299-b4f8-effbe2ec1829 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:03.940294 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7cb22259-7d64-4299-b4f8-effbe2ec1829 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:04.037648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7cb22259-7d64-4299-b4f8-effbe2ec1829 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:04.047440 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7cb22259-7d64-4299-b4f8-effbe2ec1829 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.127382 Aug 07 01:17:04.048544 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 344/710] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:03 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:05.082971 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-94250093-30e4-45d1-83a0-820fd02fd302 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:05.083692 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-94250093-30e4-45d1-83a0-820fd02fd302 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:05.091585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94250093-30e4-45d1-83a0-820fd02fd302 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:05.092081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94250093-30e4-45d1-83a0-820fd02fd302 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:05.092562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94250093-30e4-45d1-83a0-820fd02fd302 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:05.093666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94250093-30e4-45d1-83a0-820fd02fd302 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:05.094238 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94250093-30e4-45d1-83a0-820fd02fd302 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:05.094693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94250093-30e4-45d1-83a0-820fd02fd302 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:05.178106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-94250093-30e4-45d1-83a0-820fd02fd302 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:05.191828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-94250093-30e4-45d1-83a0-820fd02fd302 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.114787 Aug 07 01:17:05.192542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 367/711] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:05 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:06.220340 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cc80ccb7-a1ac-497c-85fe-6398328cfb1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:06.221675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-cc80ccb7-a1ac-497c-85fe-6398328cfb1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:06.232707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc80ccb7-a1ac-497c-85fe-6398328cfb1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:06.233366 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc80ccb7-a1ac-497c-85fe-6398328cfb1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:06.234517 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc80ccb7-a1ac-497c-85fe-6398328cfb1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:06.235583 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc80ccb7-a1ac-497c-85fe-6398328cfb1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:06.236213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc80ccb7-a1ac-497c-85fe-6398328cfb1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:06.236831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc80ccb7-a1ac-497c-85fe-6398328cfb1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:06.356063 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-cc80ccb7-a1ac-497c-85fe-6398328cfb1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:06.366844 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-cc80ccb7-a1ac-497c-85fe-6398328cfb1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.154170 Aug 07 01:17:06.367814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 345/712] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:06 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:07.402054 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1390f88e-0f1f-42a3-9579-051baa153486 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:07.403070 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1390f88e-0f1f-42a3-9579-051baa153486 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:07.415631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1390f88e-0f1f-42a3-9579-051baa153486 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:07.416558 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1390f88e-0f1f-42a3-9579-051baa153486 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:07.417408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1390f88e-0f1f-42a3-9579-051baa153486 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:07.418836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1390f88e-0f1f-42a3-9579-051baa153486 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:07.419692 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1390f88e-0f1f-42a3-9579-051baa153486 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:07.420600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1390f88e-0f1f-42a3-9579-051baa153486 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:07.497958 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1390f88e-0f1f-42a3-9579-051baa153486 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:07.507507 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1390f88e-0f1f-42a3-9579-051baa153486 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.112773 Aug 07 01:17:07.508280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 368/713] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:07 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:08.379256 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:17:08.380076 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:17:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:17:08.531861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d3a82cd9-7a23-4b48-a4ae-765d0b1d9261 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:08.532596 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d3a82cd9-7a23-4b48-a4ae-765d0b1d9261 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:08.541465 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d3a82cd9-7a23-4b48-a4ae-765d0b1d9261 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:08.541977 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d3a82cd9-7a23-4b48-a4ae-765d0b1d9261 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:08.542515 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d3a82cd9-7a23-4b48-a4ae-765d0b1d9261 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:08.543444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d3a82cd9-7a23-4b48-a4ae-765d0b1d9261 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:08.543961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d3a82cd9-7a23-4b48-a4ae-765d0b1d9261 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:08.544652 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d3a82cd9-7a23-4b48-a4ae-765d0b1d9261 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:08.645854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d3a82cd9-7a23-4b48-a4ae-765d0b1d9261 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:08.661817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d3a82cd9-7a23-4b48-a4ae-765d0b1d9261 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.133920 Aug 07 01:17:08.662892 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 346/714] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:08 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:09.702430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-68ecb46c-4679-48ae-bc4c-229401a8793b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:09.704202 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-68ecb46c-4679-48ae-bc4c-229401a8793b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:09.715022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68ecb46c-4679-48ae-bc4c-229401a8793b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:09.715898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68ecb46c-4679-48ae-bc4c-229401a8793b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:09.716732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68ecb46c-4679-48ae-bc4c-229401a8793b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:09.717981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68ecb46c-4679-48ae-bc4c-229401a8793b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:09.718833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68ecb46c-4679-48ae-bc4c-229401a8793b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:09.719608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68ecb46c-4679-48ae-bc4c-229401a8793b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:09.792518 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-68ecb46c-4679-48ae-bc4c-229401a8793b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:09.804043 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-68ecb46c-4679-48ae-bc4c-229401a8793b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.108809 Aug 07 01:17:09.805180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 369/715] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:09 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:10.839457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-957afa0d-8b66-4c36-8761-43625bb6dd2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:10.840126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-957afa0d-8b66-4c36-8761-43625bb6dd2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:10.847440 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-957afa0d-8b66-4c36-8761-43625bb6dd2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:10.847945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-957afa0d-8b66-4c36-8761-43625bb6dd2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:10.848495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-957afa0d-8b66-4c36-8761-43625bb6dd2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:10.849227 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-957afa0d-8b66-4c36-8761-43625bb6dd2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:10.849691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-957afa0d-8b66-4c36-8761-43625bb6dd2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:10.850119 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-957afa0d-8b66-4c36-8761-43625bb6dd2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:10.921177 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-957afa0d-8b66-4c36-8761-43625bb6dd2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:10.932564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-957afa0d-8b66-4c36-8761-43625bb6dd2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.096976 Aug 07 01:17:10.933164 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 347/716] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:10 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:11.959186 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ccd9e144-fcdf-4139-9baf-ef36b04adbbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:11.960328 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ccd9e144-fcdf-4139-9baf-ef36b04adbbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:11.972162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccd9e144-fcdf-4139-9baf-ef36b04adbbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:11.972966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccd9e144-fcdf-4139-9baf-ef36b04adbbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:11.973700 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccd9e144-fcdf-4139-9baf-ef36b04adbbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:11.974953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccd9e144-fcdf-4139-9baf-ef36b04adbbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:11.975738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccd9e144-fcdf-4139-9baf-ef36b04adbbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:11.976563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccd9e144-fcdf-4139-9baf-ef36b04adbbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:12.081709 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ccd9e144-fcdf-4139-9baf-ef36b04adbbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:12.093955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ccd9e144-fcdf-4139-9baf-ef36b04adbbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.142367 Aug 07 01:17:12.094850 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 370/717] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:11 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:13.133663 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-235b5ea2-8503-4b29-8e30-7c58f91441ed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:13.134806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-235b5ea2-8503-4b29-8e30-7c58f91441ed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:13.145734 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-235b5ea2-8503-4b29-8e30-7c58f91441ed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:13.146338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-235b5ea2-8503-4b29-8e30-7c58f91441ed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:13.146809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-235b5ea2-8503-4b29-8e30-7c58f91441ed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:13.147552 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-235b5ea2-8503-4b29-8e30-7c58f91441ed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:13.148108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-235b5ea2-8503-4b29-8e30-7c58f91441ed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:13.148601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-235b5ea2-8503-4b29-8e30-7c58f91441ed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:13.271400 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-235b5ea2-8503-4b29-8e30-7c58f91441ed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:13.282982 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-235b5ea2-8503-4b29-8e30-7c58f91441ed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.155878 Aug 07 01:17:13.284289 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 348/718] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:13 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:14.307057 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d2a80acb-b16d-42e3-996d-426183c4bc65 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:14.308194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d2a80acb-b16d-42e3-996d-426183c4bc65 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:14.320725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d2a80acb-b16d-42e3-996d-426183c4bc65 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:14.321574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d2a80acb-b16d-42e3-996d-426183c4bc65 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:14.322447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d2a80acb-b16d-42e3-996d-426183c4bc65 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:14.323606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d2a80acb-b16d-42e3-996d-426183c4bc65 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:14.324475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d2a80acb-b16d-42e3-996d-426183c4bc65 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:14.325240 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d2a80acb-b16d-42e3-996d-426183c4bc65 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:14.429205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d2a80acb-b16d-42e3-996d-426183c4bc65 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:14.441052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d2a80acb-b16d-42e3-996d-426183c4bc65 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.140076 Aug 07 01:17:14.441648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 371/719] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:14 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:15.386190 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6bf053f8-68a7-491f-b950-c3ee601a3d6e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:15.386834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6bf053f8-68a7-491f-b950-c3ee601a3d6e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:15.393797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6bf053f8-68a7-491f-b950-c3ee601a3d6e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:15.395426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6bf053f8-68a7-491f-b950-c3ee601a3d6e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:15.395426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6bf053f8-68a7-491f-b950-c3ee601a3d6e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:15.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6bf053f8-68a7-491f-b950-c3ee601a3d6e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:15.396503 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6bf053f8-68a7-491f-b950-c3ee601a3d6e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:15.397048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6bf053f8-68a7-491f-b950-c3ee601a3d6e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:15.480331 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-09358589-1ab7-41a7-8978-f7c46689620c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:15.481514 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-09358589-1ab7-41a7-8978-f7c46689620c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:15.491838 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09358589-1ab7-41a7-8978-f7c46689620c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:15.492373 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09358589-1ab7-41a7-8978-f7c46689620c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:15.492817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09358589-1ab7-41a7-8978-f7c46689620c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:15.494223 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09358589-1ab7-41a7-8978-f7c46689620c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:15.494712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09358589-1ab7-41a7-8978-f7c46689620c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:15.495171 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09358589-1ab7-41a7-8978-f7c46689620c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:15.508217 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.objects.instance [None req-6bf053f8-68a7-491f-b950-c3ee601a3d6e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lazy-loading 'services' on Instance uuid 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd {{(pid=117984) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:17:15.571711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-09358589-1ab7-41a7-8978-f7c46689620c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:15.583978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-09358589-1ab7-41a7-8978-f7c46689620c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.111242 Aug 07 01:17:15.584823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 372/720] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:15 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:15.648569 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6bf053f8-68a7-491f-b950-c3ee601a3d6e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "DELETE /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments/49c52ecb-fd7f-409d-b4c7-6f5bb414da0f" status: 202 len: 0 microversion: 2.1 time: 0.266153 Aug 07 01:17:15.649497 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 349/721] 10.222.0.26 () {58 vars in 1416 bytes} [Sun Aug 7 01:17:15 2022] DELETE /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments/49c52ecb-fd7f-409d-b4c7-6f5bb414da0f => generated 0 bytes in 268 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:17:15.665226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-02aaa751-8b28-4182-9925-af48a9b6e779 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:15.669531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-02aaa751-8b28-4182-9925-af48a9b6e779 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:15.684553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-02aaa751-8b28-4182-9925-af48a9b6e779 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:15.685398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-02aaa751-8b28-4182-9925-af48a9b6e779 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:15.686036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-02aaa751-8b28-4182-9925-af48a9b6e779 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:15.689076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-02aaa751-8b28-4182-9925-af48a9b6e779 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:15.690010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-02aaa751-8b28-4182-9925-af48a9b6e779 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:15.691051 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-02aaa751-8b28-4182-9925-af48a9b6e779 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:15.760298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-02aaa751-8b28-4182-9925-af48a9b6e779 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.103945 Aug 07 01:17:15.761600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 373/722] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:15 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:16.606524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b888546f-c170-431a-9142-dacbe99e1840 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:16.607500 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b888546f-c170-431a-9142-dacbe99e1840 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:16.615230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b888546f-c170-431a-9142-dacbe99e1840 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:16.615769 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b888546f-c170-431a-9142-dacbe99e1840 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:16.616266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b888546f-c170-431a-9142-dacbe99e1840 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:16.617465 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b888546f-c170-431a-9142-dacbe99e1840 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:16.618104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b888546f-c170-431a-9142-dacbe99e1840 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:16.618686 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b888546f-c170-431a-9142-dacbe99e1840 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:16.707990 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b888546f-c170-431a-9142-dacbe99e1840 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:16.716220 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b888546f-c170-431a-9142-dacbe99e1840 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.114607 Aug 07 01:17:16.716849 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 350/723] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:16 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:16.777212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ed797d97-b215-4957-9b17-138bb7010496 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:16.779516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ed797d97-b215-4957-9b17-138bb7010496 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:16.786873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed797d97-b215-4957-9b17-138bb7010496 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:16.787366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed797d97-b215-4957-9b17-138bb7010496 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:16.787806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed797d97-b215-4957-9b17-138bb7010496 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:16.788576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed797d97-b215-4957-9b17-138bb7010496 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:16.789031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed797d97-b215-4957-9b17-138bb7010496 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:16.789460 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed797d97-b215-4957-9b17-138bb7010496 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:16.828109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ed797d97-b215-4957-9b17-138bb7010496 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054835 Aug 07 01:17:16.828762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 374/724] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:16 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:17.736773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-71f72d35-7d1c-421d-b1af-b9418f9e592e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:17.737442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-71f72d35-7d1c-421d-b1af-b9418f9e592e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:17.744741 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-71f72d35-7d1c-421d-b1af-b9418f9e592e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:17.745241 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-71f72d35-7d1c-421d-b1af-b9418f9e592e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:17.745683 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-71f72d35-7d1c-421d-b1af-b9418f9e592e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:17.746409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-71f72d35-7d1c-421d-b1af-b9418f9e592e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:17.746866 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-71f72d35-7d1c-421d-b1af-b9418f9e592e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:17.747300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-71f72d35-7d1c-421d-b1af-b9418f9e592e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:17.819319 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-71f72d35-7d1c-421d-b1af-b9418f9e592e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:17.834258 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-71f72d35-7d1c-421d-b1af-b9418f9e592e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.101160 Aug 07 01:17:17.835298 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 351/725] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:17 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:17.844124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e5fe2260-f5ae-4720-ac21-a6afccf1b5b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:17.846363 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e5fe2260-f5ae-4720-ac21-a6afccf1b5b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:17.853415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5fe2260-f5ae-4720-ac21-a6afccf1b5b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:17.853897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5fe2260-f5ae-4720-ac21-a6afccf1b5b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:17.854341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5fe2260-f5ae-4720-ac21-a6afccf1b5b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:17.855106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5fe2260-f5ae-4720-ac21-a6afccf1b5b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:17.855523 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5fe2260-f5ae-4720-ac21-a6afccf1b5b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:17.855950 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5fe2260-f5ae-4720-ac21-a6afccf1b5b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:17.892551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e5fe2260-f5ae-4720-ac21-a6afccf1b5b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052128 Aug 07 01:17:17.893173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 375/726] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:17 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:18.867353 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d9746b12-63f8-4129-958b-f5866b7c1c5f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:18.868081 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d9746b12-63f8-4129-958b-f5866b7c1c5f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:18.876505 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9746b12-63f8-4129-958b-f5866b7c1c5f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:18.877207 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9746b12-63f8-4129-958b-f5866b7c1c5f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:18.877713 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9746b12-63f8-4129-958b-f5866b7c1c5f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:18.878725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9746b12-63f8-4129-958b-f5866b7c1c5f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:18.879258 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9746b12-63f8-4129-958b-f5866b7c1c5f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:18.879782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9746b12-63f8-4129-958b-f5866b7c1c5f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:18.907465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f07e5b18-7efe-4092-8aa1-8c98c33589b0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:18.910049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f07e5b18-7efe-4092-8aa1-8c98c33589b0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:18.918476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f07e5b18-7efe-4092-8aa1-8c98c33589b0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:18.919362 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f07e5b18-7efe-4092-8aa1-8c98c33589b0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:18.920193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f07e5b18-7efe-4092-8aa1-8c98c33589b0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:18.921989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f07e5b18-7efe-4092-8aa1-8c98c33589b0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:18.922822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f07e5b18-7efe-4092-8aa1-8c98c33589b0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:18.923605 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f07e5b18-7efe-4092-8aa1-8c98c33589b0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:18.973707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d9746b12-63f8-4129-958b-f5866b7c1c5f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:18.978353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f07e5b18-7efe-4092-8aa1-8c98c33589b0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.075310 Aug 07 01:17:18.979481 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 376/727] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:18 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:18.988071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d9746b12-63f8-4129-958b-f5866b7c1c5f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.124367 Aug 07 01:17:18.989365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 352/728] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:18 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:19.992865 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-47df80de-fdcf-4453-a46b-598c3afe360d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:19.995600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-47df80de-fdcf-4453-a46b-598c3afe360d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:20.004526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-47df80de-fdcf-4453-a46b-598c3afe360d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:20.004949 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-47df80de-fdcf-4453-a46b-598c3afe360d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:20.005398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-47df80de-fdcf-4453-a46b-598c3afe360d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:20.006166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-47df80de-fdcf-4453-a46b-598c3afe360d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:20.006574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-47df80de-fdcf-4453-a46b-598c3afe360d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:20.007005 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-47df80de-fdcf-4453-a46b-598c3afe360d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:20.025006 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4ad8c305-4cd0-43a7-b79b-2387985a8e32 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:20.025984 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4ad8c305-4cd0-43a7-b79b-2387985a8e32 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:20.037953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ad8c305-4cd0-43a7-b79b-2387985a8e32 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:20.038792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ad8c305-4cd0-43a7-b79b-2387985a8e32 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:20.039509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ad8c305-4cd0-43a7-b79b-2387985a8e32 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:20.040472 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ad8c305-4cd0-43a7-b79b-2387985a8e32 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:20.041069 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ad8c305-4cd0-43a7-b79b-2387985a8e32 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:20.042024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4ad8c305-4cd0-43a7-b79b-2387985a8e32 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:20.052083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-47df80de-fdcf-4453-a46b-598c3afe360d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064102 Aug 07 01:17:20.053378 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 377/729] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:19 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:20.129827 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4ad8c305-4cd0-43a7-b79b-2387985a8e32 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:20.139296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4ad8c305-4cd0-43a7-b79b-2387985a8e32 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.119183 Aug 07 01:17:20.140113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 353/730] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:20 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:21.071864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fbcb92b4-bab4-4455-a8e0-724dc8d1d1b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:21.076438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fbcb92b4-bab4-4455-a8e0-724dc8d1d1b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:21.089751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fbcb92b4-bab4-4455-a8e0-724dc8d1d1b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:21.090635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fbcb92b4-bab4-4455-a8e0-724dc8d1d1b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:21.091441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fbcb92b4-bab4-4455-a8e0-724dc8d1d1b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:21.092869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fbcb92b4-bab4-4455-a8e0-724dc8d1d1b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:21.094273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fbcb92b4-bab4-4455-a8e0-724dc8d1d1b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:21.094997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fbcb92b4-bab4-4455-a8e0-724dc8d1d1b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:21.160768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fbcb92b4-bab4-4455-a8e0-724dc8d1d1b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.097417 Aug 07 01:17:21.162778 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 378/731] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:21 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:21.225251 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-eecb31f4-28b5-487d-8a36-1d8c4f5f72fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:21.226072 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-eecb31f4-28b5-487d-8a36-1d8c4f5f72fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:21.236204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eecb31f4-28b5-487d-8a36-1d8c4f5f72fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:21.237170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eecb31f4-28b5-487d-8a36-1d8c4f5f72fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:21.237709 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eecb31f4-28b5-487d-8a36-1d8c4f5f72fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:21.238988 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eecb31f4-28b5-487d-8a36-1d8c4f5f72fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:21.239815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eecb31f4-28b5-487d-8a36-1d8c4f5f72fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:21.240600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eecb31f4-28b5-487d-8a36-1d8c4f5f72fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:21.361652 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-eecb31f4-28b5-487d-8a36-1d8c4f5f72fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:21.371809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-eecb31f4-28b5-487d-8a36-1d8c4f5f72fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.151229 Aug 07 01:17:21.372478 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 354/732] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:21 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:22.176690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-20c42d68-cb2c-4f19-ac68-b11e217a3063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:22.180161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-20c42d68-cb2c-4f19-ac68-b11e217a3063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:22.191148 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20c42d68-cb2c-4f19-ac68-b11e217a3063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:22.192021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20c42d68-cb2c-4f19-ac68-b11e217a3063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:22.192948 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20c42d68-cb2c-4f19-ac68-b11e217a3063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:22.194199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20c42d68-cb2c-4f19-ac68-b11e217a3063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:22.195041 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20c42d68-cb2c-4f19-ac68-b11e217a3063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:22.195834 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20c42d68-cb2c-4f19-ac68-b11e217a3063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:22.252068 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-20c42d68-cb2c-4f19-ac68-b11e217a3063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080405 Aug 07 01:17:22.252747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 379/733] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:22 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:22.399806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0aafffa7-1a30-41b4-a088-d240ea5c91cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:22.400678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0aafffa7-1a30-41b4-a088-d240ea5c91cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:22.409155 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0aafffa7-1a30-41b4-a088-d240ea5c91cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:22.410898 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0aafffa7-1a30-41b4-a088-d240ea5c91cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:22.410898 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0aafffa7-1a30-41b4-a088-d240ea5c91cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:22.411244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0aafffa7-1a30-41b4-a088-d240ea5c91cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:22.411532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0aafffa7-1a30-41b4-a088-d240ea5c91cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:22.411951 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0aafffa7-1a30-41b4-a088-d240ea5c91cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:22.497405 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0aafffa7-1a30-41b4-a088-d240ea5c91cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:22.505514 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:17:22.515649 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0aafffa7-1a30-41b4-a088-d240ea5c91cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.121257 Aug 07 01:17:22.516618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 355/734] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:22 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:23.269822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8758c495-1dca-4155-85ce-a5f6faa28c13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:23.272657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8758c495-1dca-4155-85ce-a5f6faa28c13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:23.281130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8758c495-1dca-4155-85ce-a5f6faa28c13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:23.281784 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8758c495-1dca-4155-85ce-a5f6faa28c13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:23.282368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8758c495-1dca-4155-85ce-a5f6faa28c13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:23.283271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8758c495-1dca-4155-85ce-a5f6faa28c13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:23.283843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8758c495-1dca-4155-85ce-a5f6faa28c13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:23.284461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8758c495-1dca-4155-85ce-a5f6faa28c13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:23.315730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:17:23.341604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8758c495-1dca-4155-85ce-a5f6faa28c13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.077157 Aug 07 01:17:23.342277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 380/735] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:23 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:23.553862 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d00c155d-f887-4165-bb37-c67a09230f53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:23.555104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d00c155d-f887-4165-bb37-c67a09230f53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:23.568132 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d00c155d-f887-4165-bb37-c67a09230f53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:23.568987 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d00c155d-f887-4165-bb37-c67a09230f53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:23.569730 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d00c155d-f887-4165-bb37-c67a09230f53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:23.570564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d00c155d-f887-4165-bb37-c67a09230f53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:23.571334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d00c155d-f887-4165-bb37-c67a09230f53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:23.572129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d00c155d-f887-4165-bb37-c67a09230f53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:23.670146 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d00c155d-f887-4165-bb37-c67a09230f53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:23.679169 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d00c155d-f887-4165-bb37-c67a09230f53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.134052 Aug 07 01:17:23.679759 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 356/736] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:23 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:24.360335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-191a86ce-7785-4874-8306-3de3de14f96b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:24.364173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-191a86ce-7785-4874-8306-3de3de14f96b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:24.377422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-191a86ce-7785-4874-8306-3de3de14f96b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:24.378277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-191a86ce-7785-4874-8306-3de3de14f96b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:24.379112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-191a86ce-7785-4874-8306-3de3de14f96b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:24.380651 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-191a86ce-7785-4874-8306-3de3de14f96b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:24.381478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-191a86ce-7785-4874-8306-3de3de14f96b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:24.382324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-191a86ce-7785-4874-8306-3de3de14f96b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:24.451653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-191a86ce-7785-4874-8306-3de3de14f96b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.099468 Aug 07 01:17:24.453570 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 381/737] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:24 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:24.708988 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d763f609-8a13-4f51-9048-4258a5e73d0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:24.709962 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d763f609-8a13-4f51-9048-4258a5e73d0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:24.723011 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d763f609-8a13-4f51-9048-4258a5e73d0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:24.723867 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d763f609-8a13-4f51-9048-4258a5e73d0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:24.724732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d763f609-8a13-4f51-9048-4258a5e73d0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:24.726096 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d763f609-8a13-4f51-9048-4258a5e73d0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:24.726981 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d763f609-8a13-4f51-9048-4258a5e73d0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:24.727770 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d763f609-8a13-4f51-9048-4258a5e73d0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:24.848357 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d763f609-8a13-4f51-9048-4258a5e73d0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:24.864522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d763f609-8a13-4f51-9048-4258a5e73d0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.163304 Aug 07 01:17:24.865648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 357/738] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:24 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:25.471036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dc64bfc7-e049-41d0-a20c-fc6c56b5338b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:25.473705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dc64bfc7-e049-41d0-a20c-fc6c56b5338b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:25.482892 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc64bfc7-e049-41d0-a20c-fc6c56b5338b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:25.483603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc64bfc7-e049-41d0-a20c-fc6c56b5338b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:25.484221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc64bfc7-e049-41d0-a20c-fc6c56b5338b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:25.485150 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc64bfc7-e049-41d0-a20c-fc6c56b5338b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:25.486120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc64bfc7-e049-41d0-a20c-fc6c56b5338b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:25.486683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc64bfc7-e049-41d0-a20c-fc6c56b5338b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:25.532029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dc64bfc7-e049-41d0-a20c-fc6c56b5338b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066161 Aug 07 01:17:25.532821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 382/739] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:25 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:25.898970 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0ed82d87-f86e-432f-992f-0e7c3650c23f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:25.899859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0ed82d87-f86e-432f-992f-0e7c3650c23f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:25.909331 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ed82d87-f86e-432f-992f-0e7c3650c23f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:25.909882 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ed82d87-f86e-432f-992f-0e7c3650c23f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:25.910387 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ed82d87-f86e-432f-992f-0e7c3650c23f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:25.911200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ed82d87-f86e-432f-992f-0e7c3650c23f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:25.911730 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ed82d87-f86e-432f-992f-0e7c3650c23f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:25.912204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0ed82d87-f86e-432f-992f-0e7c3650c23f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:26.025759 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0ed82d87-f86e-432f-992f-0e7c3650c23f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:26.034837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0ed82d87-f86e-432f-992f-0e7c3650c23f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.140065 Aug 07 01:17:26.035501 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 358/740] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:25 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:26.546888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-68d4736b-5804-48b0-8d4c-1ae0c320e01a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:26.549055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-68d4736b-5804-48b0-8d4c-1ae0c320e01a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:26.556214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68d4736b-5804-48b0-8d4c-1ae0c320e01a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:26.556884 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68d4736b-5804-48b0-8d4c-1ae0c320e01a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:26.557421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68d4736b-5804-48b0-8d4c-1ae0c320e01a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:26.558269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68d4736b-5804-48b0-8d4c-1ae0c320e01a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:26.558827 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68d4736b-5804-48b0-8d4c-1ae0c320e01a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:26.559351 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68d4736b-5804-48b0-8d4c-1ae0c320e01a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:26.604576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-68d4736b-5804-48b0-8d4c-1ae0c320e01a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062199 Aug 07 01:17:26.605514 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 383/741] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:26 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:27.058298 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1f985390-fee7-4ab5-8871-9925b6de1a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:27.059333 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1f985390-fee7-4ab5-8871-9925b6de1a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:27.067785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f985390-fee7-4ab5-8871-9925b6de1a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:27.068363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f985390-fee7-4ab5-8871-9925b6de1a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:27.068839 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f985390-fee7-4ab5-8871-9925b6de1a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:27.069621 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f985390-fee7-4ab5-8871-9925b6de1a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:27.070164 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f985390-fee7-4ab5-8871-9925b6de1a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:27.070590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f985390-fee7-4ab5-8871-9925b6de1a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:27.178205 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1f985390-fee7-4ab5-8871-9925b6de1a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:27.193043 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1f985390-fee7-4ab5-8871-9925b6de1a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.138817 Aug 07 01:17:27.194354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 359/742] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:27 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:27.621640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a4a2d8f6-28ec-4e30-b30c-86d13eb5cfd6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:27.625054 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a4a2d8f6-28ec-4e30-b30c-86d13eb5cfd6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:27.633105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4a2d8f6-28ec-4e30-b30c-86d13eb5cfd6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:27.633667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4a2d8f6-28ec-4e30-b30c-86d13eb5cfd6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:27.634204 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4a2d8f6-28ec-4e30-b30c-86d13eb5cfd6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:27.634994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4a2d8f6-28ec-4e30-b30c-86d13eb5cfd6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:27.635505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4a2d8f6-28ec-4e30-b30c-86d13eb5cfd6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:27.635981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4a2d8f6-28ec-4e30-b30c-86d13eb5cfd6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:27.673558 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a4a2d8f6-28ec-4e30-b30c-86d13eb5cfd6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056369 Aug 07 01:17:27.674151 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 384/743] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:27 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:28.234822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d327b4af-e627-47fd-b87c-fc1083488641 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:28.235638 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d327b4af-e627-47fd-b87c-fc1083488641 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:28.243521 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d327b4af-e627-47fd-b87c-fc1083488641 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:28.244063 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d327b4af-e627-47fd-b87c-fc1083488641 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:28.244691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d327b4af-e627-47fd-b87c-fc1083488641 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:28.245339 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d327b4af-e627-47fd-b87c-fc1083488641 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:28.245805 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d327b4af-e627-47fd-b87c-fc1083488641 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:28.246244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d327b4af-e627-47fd-b87c-fc1083488641 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:28.334558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d327b4af-e627-47fd-b87c-fc1083488641 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:28.345034 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d327b4af-e627-47fd-b87c-fc1083488641 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.115278 Aug 07 01:17:28.345640 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 360/744] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:28 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:28.687693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cb5becce-910b-4423-bb32-97a9b5e09993 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:28.690016 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cb5becce-910b-4423-bb32-97a9b5e09993 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:28.697188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cb5becce-910b-4423-bb32-97a9b5e09993 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:28.697688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cb5becce-910b-4423-bb32-97a9b5e09993 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:28.698140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cb5becce-910b-4423-bb32-97a9b5e09993 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:28.698887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cb5becce-910b-4423-bb32-97a9b5e09993 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:28.699346 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cb5becce-910b-4423-bb32-97a9b5e09993 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:28.699777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cb5becce-910b-4423-bb32-97a9b5e09993 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:28.741503 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cb5becce-910b-4423-bb32-97a9b5e09993 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058545 Aug 07 01:17:28.742304 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 385/745] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:28 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:29.369203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b93a5fe9-22b4-4607-8b22-02c752771a94 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:29.370006 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b93a5fe9-22b4-4607-8b22-02c752771a94 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:29.372662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=46,nova_api:UPDATE=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:17:29.379991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b93a5fe9-22b4-4607-8b22-02c752771a94 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:29.380543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b93a5fe9-22b4-4607-8b22-02c752771a94 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:29.380987 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b93a5fe9-22b4-4607-8b22-02c752771a94 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:29.381711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b93a5fe9-22b4-4607-8b22-02c752771a94 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:29.382170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b93a5fe9-22b4-4607-8b22-02c752771a94 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:29.382597 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b93a5fe9-22b4-4607-8b22-02c752771a94 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:29.464967 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b93a5fe9-22b4-4607-8b22-02c752771a94 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:29.474283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b93a5fe9-22b4-4607-8b22-02c752771a94 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.108994 Aug 07 01:17:29.475027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 361/746] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:29 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:29.755825 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8924eea1-0c83-4806-bde0-28a03b0fe7a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:29.758651 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8924eea1-0c83-4806-bde0-28a03b0fe7a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:29.762560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=45 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:17:29.770197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8924eea1-0c83-4806-bde0-28a03b0fe7a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:29.770737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8924eea1-0c83-4806-bde0-28a03b0fe7a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:29.771184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8924eea1-0c83-4806-bde0-28a03b0fe7a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:29.771948 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8924eea1-0c83-4806-bde0-28a03b0fe7a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:29.772499 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8924eea1-0c83-4806-bde0-28a03b0fe7a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:29.772937 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8924eea1-0c83-4806-bde0-28a03b0fe7a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:29.822178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8924eea1-0c83-4806-bde0-28a03b0fe7a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.070598 Aug 07 01:17:29.823244 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 386/747] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:29 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:30.496482 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bb5e5285-9935-459f-b8db-b09f5dc2dcf3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:30.497180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bb5e5285-9935-459f-b8db-b09f5dc2dcf3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:30.506626 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bb5e5285-9935-459f-b8db-b09f5dc2dcf3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:30.507391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bb5e5285-9935-459f-b8db-b09f5dc2dcf3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:30.507907 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bb5e5285-9935-459f-b8db-b09f5dc2dcf3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:30.508751 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bb5e5285-9935-459f-b8db-b09f5dc2dcf3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:30.509355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bb5e5285-9935-459f-b8db-b09f5dc2dcf3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:30.509831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bb5e5285-9935-459f-b8db-b09f5dc2dcf3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:30.587070 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bb5e5285-9935-459f-b8db-b09f5dc2dcf3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:30.600901 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bb5e5285-9935-459f-b8db-b09f5dc2dcf3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.108613 Aug 07 01:17:30.602079 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 362/748] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:30 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:30.841594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-44aaa51e-7a5e-43ba-b90d-75e42fad019a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:30.844928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-44aaa51e-7a5e-43ba-b90d-75e42fad019a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:30.853801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44aaa51e-7a5e-43ba-b90d-75e42fad019a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:30.854517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44aaa51e-7a5e-43ba-b90d-75e42fad019a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:30.855122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44aaa51e-7a5e-43ba-b90d-75e42fad019a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:30.856440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44aaa51e-7a5e-43ba-b90d-75e42fad019a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:30.857277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44aaa51e-7a5e-43ba-b90d-75e42fad019a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:30.858043 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44aaa51e-7a5e-43ba-b90d-75e42fad019a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:30.916387 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-44aaa51e-7a5e-43ba-b90d-75e42fad019a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.079952 Aug 07 01:17:30.917393 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 387/749] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:30 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:31.625442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-43e7526b-4266-4dce-a391-b0875355efb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:31.626131 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-43e7526b-4266-4dce-a391-b0875355efb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:31.633472 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-43e7526b-4266-4dce-a391-b0875355efb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:31.633987 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-43e7526b-4266-4dce-a391-b0875355efb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:31.634433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-43e7526b-4266-4dce-a391-b0875355efb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:31.635160 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-43e7526b-4266-4dce-a391-b0875355efb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:31.635622 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-43e7526b-4266-4dce-a391-b0875355efb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:31.636105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-43e7526b-4266-4dce-a391-b0875355efb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:31.715211 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-43e7526b-4266-4dce-a391-b0875355efb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:31.731685 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-43e7526b-4266-4dce-a391-b0875355efb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.111031 Aug 07 01:17:31.732306 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 363/750] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:31 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:31.933386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1ec9c6a1-1b9e-405f-93ff-9567fd4a7e8c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:31.935993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1ec9c6a1-1b9e-405f-93ff-9567fd4a7e8c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:31.943213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ec9c6a1-1b9e-405f-93ff-9567fd4a7e8c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:31.943709 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ec9c6a1-1b9e-405f-93ff-9567fd4a7e8c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:31.944152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ec9c6a1-1b9e-405f-93ff-9567fd4a7e8c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:31.944916 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ec9c6a1-1b9e-405f-93ff-9567fd4a7e8c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:31.945375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ec9c6a1-1b9e-405f-93ff-9567fd4a7e8c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:31.945804 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ec9c6a1-1b9e-405f-93ff-9567fd4a7e8c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:31.984882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1ec9c6a1-1b9e-405f-93ff-9567fd4a7e8c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055847 Aug 07 01:17:31.985549 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 388/751] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:31 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:32.761266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7a0fe6e8-afff-48f9-aa1c-ea45f46bb28a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:32.762284 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7a0fe6e8-afff-48f9-aa1c-ea45f46bb28a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:32.770889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a0fe6e8-afff-48f9-aa1c-ea45f46bb28a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:32.771546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a0fe6e8-afff-48f9-aa1c-ea45f46bb28a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:32.771836 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a0fe6e8-afff-48f9-aa1c-ea45f46bb28a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:32.775342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a0fe6e8-afff-48f9-aa1c-ea45f46bb28a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:32.775342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a0fe6e8-afff-48f9-aa1c-ea45f46bb28a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:32.775342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a0fe6e8-afff-48f9-aa1c-ea45f46bb28a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:32.845640 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7a0fe6e8-afff-48f9-aa1c-ea45f46bb28a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:32.854997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7a0fe6e8-afff-48f9-aa1c-ea45f46bb28a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097786 Aug 07 01:17:32.855691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 364/752] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:32 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:33.001199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-659fac84-ab84-44e3-a665-2e29fa5a576f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:33.003530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-659fac84-ab84-44e3-a665-2e29fa5a576f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:33.011094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-659fac84-ab84-44e3-a665-2e29fa5a576f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:33.011683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-659fac84-ab84-44e3-a665-2e29fa5a576f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:33.012199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-659fac84-ab84-44e3-a665-2e29fa5a576f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:33.013061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-659fac84-ab84-44e3-a665-2e29fa5a576f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:33.013575 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-659fac84-ab84-44e3-a665-2e29fa5a576f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:33.014216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-659fac84-ab84-44e3-a665-2e29fa5a576f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:33.057551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-659fac84-ab84-44e3-a665-2e29fa5a576f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059555 Aug 07 01:17:33.057551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 389/753] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:32 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:33.879587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f6c0efd9-fe00-4904-b25d-90c0a7668308 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:33.880344 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f6c0efd9-fe00-4904-b25d-90c0a7668308 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:33.887731 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6c0efd9-fe00-4904-b25d-90c0a7668308 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:33.888222 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6c0efd9-fe00-4904-b25d-90c0a7668308 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:33.888764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6c0efd9-fe00-4904-b25d-90c0a7668308 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:33.889495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6c0efd9-fe00-4904-b25d-90c0a7668308 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:33.890580 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6c0efd9-fe00-4904-b25d-90c0a7668308 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:33.890980 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6c0efd9-fe00-4904-b25d-90c0a7668308 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:33.978187 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f6c0efd9-fe00-4904-b25d-90c0a7668308 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:33.988775 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f6c0efd9-fe00-4904-b25d-90c0a7668308 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.112794 Aug 07 01:17:33.989665 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 365/754] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:33 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:34.079563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4f3c3cb9-d103-4233-b3a1-5ffcb0930e58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:34.083327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4f3c3cb9-d103-4233-b3a1-5ffcb0930e58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:34.096470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4f3c3cb9-d103-4233-b3a1-5ffcb0930e58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:34.097347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4f3c3cb9-d103-4233-b3a1-5ffcb0930e58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:34.098167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4f3c3cb9-d103-4233-b3a1-5ffcb0930e58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:34.099478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4f3c3cb9-d103-4233-b3a1-5ffcb0930e58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:34.100359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4f3c3cb9-d103-4233-b3a1-5ffcb0930e58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:34.101148 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4f3c3cb9-d103-4233-b3a1-5ffcb0930e58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:34.142680 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4f3c3cb9-d103-4233-b3a1-5ffcb0930e58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.071438 Aug 07 01:17:34.143873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 390/755] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:34 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:35.010438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bebbae73-3405-423c-9f31-ffcad60cd6b8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:35.011164 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bebbae73-3405-423c-9f31-ffcad60cd6b8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:35.018321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bebbae73-3405-423c-9f31-ffcad60cd6b8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:35.018816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bebbae73-3405-423c-9f31-ffcad60cd6b8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:35.019263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bebbae73-3405-423c-9f31-ffcad60cd6b8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:35.020004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bebbae73-3405-423c-9f31-ffcad60cd6b8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:35.020523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bebbae73-3405-423c-9f31-ffcad60cd6b8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:35.020960 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bebbae73-3405-423c-9f31-ffcad60cd6b8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:35.095048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bebbae73-3405-423c-9f31-ffcad60cd6b8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:35.104218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bebbae73-3405-423c-9f31-ffcad60cd6b8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097665 Aug 07 01:17:35.105256 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 366/756] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:35 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:35.156814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8ce5b86a-84aa-4919-923a-2e0ff1dbafd8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:35.159100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8ce5b86a-84aa-4919-923a-2e0ff1dbafd8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:35.166406 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8ce5b86a-84aa-4919-923a-2e0ff1dbafd8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:35.166989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8ce5b86a-84aa-4919-923a-2e0ff1dbafd8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:35.167438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8ce5b86a-84aa-4919-923a-2e0ff1dbafd8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:35.168208 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8ce5b86a-84aa-4919-923a-2e0ff1dbafd8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:35.168706 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8ce5b86a-84aa-4919-923a-2e0ff1dbafd8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:35.169144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8ce5b86a-84aa-4919-923a-2e0ff1dbafd8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:35.205867 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8ce5b86a-84aa-4919-923a-2e0ff1dbafd8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053790 Aug 07 01:17:35.206443 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 391/757] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:35 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:36.127506 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d1df5aa8-2af4-4d01-b2d9-73de4eaa9cb9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:36.128150 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d1df5aa8-2af4-4d01-b2d9-73de4eaa9cb9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:36.135074 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1df5aa8-2af4-4d01-b2d9-73de4eaa9cb9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:36.135570 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1df5aa8-2af4-4d01-b2d9-73de4eaa9cb9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:36.136008 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1df5aa8-2af4-4d01-b2d9-73de4eaa9cb9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:36.136773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1df5aa8-2af4-4d01-b2d9-73de4eaa9cb9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:36.137231 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1df5aa8-2af4-4d01-b2d9-73de4eaa9cb9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:36.137662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1df5aa8-2af4-4d01-b2d9-73de4eaa9cb9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:36.217994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-207e8549-8179-4dd0-bcf6-198f40c43ee3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:36.220521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-207e8549-8179-4dd0-bcf6-198f40c43ee3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:36.227640 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d1df5aa8-2af4-4d01-b2d9-73de4eaa9cb9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:36.229066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-207e8549-8179-4dd0-bcf6-198f40c43ee3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:36.229650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-207e8549-8179-4dd0-bcf6-198f40c43ee3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:36.230210 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-207e8549-8179-4dd0-bcf6-198f40c43ee3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:36.230838 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-207e8549-8179-4dd0-bcf6-198f40c43ee3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:36.231300 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-207e8549-8179-4dd0-bcf6-198f40c43ee3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:36.231728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-207e8549-8179-4dd0-bcf6-198f40c43ee3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:36.237083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d1df5aa8-2af4-4d01-b2d9-73de4eaa9cb9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.114162 Aug 07 01:17:36.237803 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 367/758] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:36 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:36.267768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-207e8549-8179-4dd0-bcf6-198f40c43ee3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053752 Aug 07 01:17:36.268401 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 392/759] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:36 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:37.259094 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-276744e0-a1e9-4ef1-a9fd-1d6f9cfff95c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:37.259998 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-276744e0-a1e9-4ef1-a9fd-1d6f9cfff95c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:37.268368 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-276744e0-a1e9-4ef1-a9fd-1d6f9cfff95c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:37.268872 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-276744e0-a1e9-4ef1-a9fd-1d6f9cfff95c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:37.269311 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-276744e0-a1e9-4ef1-a9fd-1d6f9cfff95c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:37.270035 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-276744e0-a1e9-4ef1-a9fd-1d6f9cfff95c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:37.270492 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-276744e0-a1e9-4ef1-a9fd-1d6f9cfff95c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:37.275492 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-276744e0-a1e9-4ef1-a9fd-1d6f9cfff95c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:37.291968 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cab98193-a216-4956-abfa-97239343f54c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:37.296087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cab98193-a216-4956-abfa-97239343f54c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:37.307859 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cab98193-a216-4956-abfa-97239343f54c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:37.308652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cab98193-a216-4956-abfa-97239343f54c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:37.309851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cab98193-a216-4956-abfa-97239343f54c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:37.310725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cab98193-a216-4956-abfa-97239343f54c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:37.311565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cab98193-a216-4956-abfa-97239343f54c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:37.312849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cab98193-a216-4956-abfa-97239343f54c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:37.357597 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-276744e0-a1e9-4ef1-a9fd-1d6f9cfff95c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:37.368641 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cab98193-a216-4956-abfa-97239343f54c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.083622 Aug 07 01:17:37.369838 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 393/760] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:37 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:37.372221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-276744e0-a1e9-4ef1-a9fd-1d6f9cfff95c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.116885 Aug 07 01:17:37.373297 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 368/761] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:37 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:38.387112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5bfa1f9f-73de-48ee-a8ae-443a019f0164 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:38.389707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5bfa1f9f-73de-48ee-a8ae-443a019f0164 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:38.398484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bfa1f9f-73de-48ee-a8ae-443a019f0164 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:38.399090 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bfa1f9f-73de-48ee-a8ae-443a019f0164 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:38.399773 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bfa1f9f-73de-48ee-a8ae-443a019f0164 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:38.400829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bfa1f9f-73de-48ee-a8ae-443a019f0164 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:38.401427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bfa1f9f-73de-48ee-a8ae-443a019f0164 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:38.401935 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bfa1f9f-73de-48ee-a8ae-443a019f0164 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:38.407328 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c4e7311e-be28-4476-8ae6-ed6b88c8f978 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:38.408511 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c4e7311e-be28-4476-8ae6-ed6b88c8f978 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:38.421056 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4e7311e-be28-4476-8ae6-ed6b88c8f978 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:38.421925 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4e7311e-be28-4476-8ae6-ed6b88c8f978 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:38.422618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4e7311e-be28-4476-8ae6-ed6b88c8f978 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:38.423797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4e7311e-be28-4476-8ae6-ed6b88c8f978 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:38.424626 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4e7311e-be28-4476-8ae6-ed6b88c8f978 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:38.425383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4e7311e-be28-4476-8ae6-ed6b88c8f978 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:38.453172 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5bfa1f9f-73de-48ee-a8ae-443a019f0164 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.070204 Aug 07 01:17:38.454431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 394/762] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:38 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:38.519999 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c4e7311e-be28-4476-8ae6-ed6b88c8f978 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:38.534286 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c4e7311e-be28-4476-8ae6-ed6b88c8f978 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.135916 Aug 07 01:17:38.535244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 369/763] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:38 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:39.471895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4436e464-0db4-4cb3-8ed7-9d79dab64815 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:39.474544 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4436e464-0db4-4cb3-8ed7-9d79dab64815 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:39.482735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4436e464-0db4-4cb3-8ed7-9d79dab64815 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:39.483241 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4436e464-0db4-4cb3-8ed7-9d79dab64815 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:39.484457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4436e464-0db4-4cb3-8ed7-9d79dab64815 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:39.485231 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4436e464-0db4-4cb3-8ed7-9d79dab64815 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:39.485717 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4436e464-0db4-4cb3-8ed7-9d79dab64815 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:39.486236 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4436e464-0db4-4cb3-8ed7-9d79dab64815 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:39.527615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4436e464-0db4-4cb3-8ed7-9d79dab64815 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062093 Aug 07 01:17:39.528226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 395/764] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:39 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:39.568125 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ff08bb8d-f9d0-483c-973f-b0d2491ef4c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:39.568942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ff08bb8d-f9d0-483c-973f-b0d2491ef4c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:39.577947 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff08bb8d-f9d0-483c-973f-b0d2491ef4c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:39.578518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff08bb8d-f9d0-483c-973f-b0d2491ef4c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:39.579011 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff08bb8d-f9d0-483c-973f-b0d2491ef4c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:39.579867 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff08bb8d-f9d0-483c-973f-b0d2491ef4c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:39.580438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff08bb8d-f9d0-483c-973f-b0d2491ef4c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:39.580909 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff08bb8d-f9d0-483c-973f-b0d2491ef4c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:39.673236 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ff08bb8d-f9d0-483c-973f-b0d2491ef4c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:39.683078 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ff08bb8d-f9d0-483c-973f-b0d2491ef4c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.119804 Aug 07 01:17:39.683834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 370/765] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:39 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:40.547934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dfe33fef-ece7-4ef2-b6c8-b3d5097eb4d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:40.549913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dfe33fef-ece7-4ef2-b6c8-b3d5097eb4d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:40.558003 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dfe33fef-ece7-4ef2-b6c8-b3d5097eb4d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:40.558764 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dfe33fef-ece7-4ef2-b6c8-b3d5097eb4d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:40.559348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dfe33fef-ece7-4ef2-b6c8-b3d5097eb4d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:40.560217 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dfe33fef-ece7-4ef2-b6c8-b3d5097eb4d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:40.560957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dfe33fef-ece7-4ef2-b6c8-b3d5097eb4d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:40.561548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dfe33fef-ece7-4ef2-b6c8-b3d5097eb4d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:40.604980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dfe33fef-ece7-4ef2-b6c8-b3d5097eb4d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064024 Aug 07 01:17:40.605645 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 396/766] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:40 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:40.723257 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6803450c-87b1-46bf-a977-55201490210f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:40.724439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6803450c-87b1-46bf-a977-55201490210f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:40.737850 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6803450c-87b1-46bf-a977-55201490210f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:40.738718 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6803450c-87b1-46bf-a977-55201490210f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:40.739516 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6803450c-87b1-46bf-a977-55201490210f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:40.740882 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6803450c-87b1-46bf-a977-55201490210f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:40.741702 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6803450c-87b1-46bf-a977-55201490210f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:40.742483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6803450c-87b1-46bf-a977-55201490210f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:40.838679 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6803450c-87b1-46bf-a977-55201490210f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:40.854142 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6803450c-87b1-46bf-a977-55201490210f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.139054 Aug 07 01:17:40.855235 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 371/767] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:40 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:41.627909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e350e64a-7349-4cbc-a6b6-fcc2480b0b19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:41.631791 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e350e64a-7349-4cbc-a6b6-fcc2480b0b19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:41.644963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e350e64a-7349-4cbc-a6b6-fcc2480b0b19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:41.645788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e350e64a-7349-4cbc-a6b6-fcc2480b0b19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:41.646615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e350e64a-7349-4cbc-a6b6-fcc2480b0b19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:41.647956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e350e64a-7349-4cbc-a6b6-fcc2480b0b19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:41.648873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e350e64a-7349-4cbc-a6b6-fcc2480b0b19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:41.649681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e350e64a-7349-4cbc-a6b6-fcc2480b0b19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:41.657853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:17:41.716728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e350e64a-7349-4cbc-a6b6-fcc2480b0b19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.097359 Aug 07 01:17:41.718116 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 397/768] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:41 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:41.894531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9a797e81-2894-4956-bd33-9b03d36795c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:41.895378 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9a797e81-2894-4956-bd33-9b03d36795c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:41.904080 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a797e81-2894-4956-bd33-9b03d36795c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:41.904748 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a797e81-2894-4956-bd33-9b03d36795c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:41.905212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a797e81-2894-4956-bd33-9b03d36795c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:41.906183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a797e81-2894-4956-bd33-9b03d36795c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:41.906965 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a797e81-2894-4956-bd33-9b03d36795c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:41.907608 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a797e81-2894-4956-bd33-9b03d36795c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:41.938316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:17:42.014651 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9a797e81-2894-4956-bd33-9b03d36795c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:42.030478 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9a797e81-2894-4956-bd33-9b03d36795c2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.144181 Aug 07 01:17:42.031391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 372/769] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:41 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:42.738747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-84375d3a-61db-4411-a2de-e588da4d712c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:42.741704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-84375d3a-61db-4411-a2de-e588da4d712c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:42.752992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-84375d3a-61db-4411-a2de-e588da4d712c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:42.753561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-84375d3a-61db-4411-a2de-e588da4d712c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:42.754129 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-84375d3a-61db-4411-a2de-e588da4d712c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:42.755455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-84375d3a-61db-4411-a2de-e588da4d712c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:42.756301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-84375d3a-61db-4411-a2de-e588da4d712c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:42.757044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-84375d3a-61db-4411-a2de-e588da4d712c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:42.814718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-84375d3a-61db-4411-a2de-e588da4d712c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.083379 Aug 07 01:17:42.815756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 398/770] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:42 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:43.058934 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-35d3fcc9-9ca9-45e7-ae6f-d7a0788e3a89 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:43.059586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-35d3fcc9-9ca9-45e7-ae6f-d7a0788e3a89 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:43.067089 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-35d3fcc9-9ca9-45e7-ae6f-d7a0788e3a89 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:43.067590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-35d3fcc9-9ca9-45e7-ae6f-d7a0788e3a89 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:43.068032 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-35d3fcc9-9ca9-45e7-ae6f-d7a0788e3a89 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:43.068906 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-35d3fcc9-9ca9-45e7-ae6f-d7a0788e3a89 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:43.069371 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-35d3fcc9-9ca9-45e7-ae6f-d7a0788e3a89 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:43.069801 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-35d3fcc9-9ca9-45e7-ae6f-d7a0788e3a89 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:43.168218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-35d3fcc9-9ca9-45e7-ae6f-d7a0788e3a89 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:43.177113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-35d3fcc9-9ca9-45e7-ae6f-d7a0788e3a89 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.122360 Aug 07 01:17:43.177836 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 373/771] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:43 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:43.838858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fcc56167-535a-43b5-a241-f62d51b793c6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:43.847352 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fcc56167-535a-43b5-a241-f62d51b793c6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:43.860006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcc56167-535a-43b5-a241-f62d51b793c6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:43.860917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcc56167-535a-43b5-a241-f62d51b793c6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:43.861693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcc56167-535a-43b5-a241-f62d51b793c6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:43.862904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcc56167-535a-43b5-a241-f62d51b793c6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:43.863676 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcc56167-535a-43b5-a241-f62d51b793c6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:43.864478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcc56167-535a-43b5-a241-f62d51b793c6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:43.927315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fcc56167-535a-43b5-a241-f62d51b793c6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.097322 Aug 07 01:17:43.928446 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 399/772] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:43 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:44.206129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0503da49-9e18-4805-87da-490ea52a5a90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:44.207772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0503da49-9e18-4805-87da-490ea52a5a90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:44.218525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0503da49-9e18-4805-87da-490ea52a5a90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:44.219406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0503da49-9e18-4805-87da-490ea52a5a90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:44.220272 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0503da49-9e18-4805-87da-490ea52a5a90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:44.221539 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0503da49-9e18-4805-87da-490ea52a5a90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:44.222365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0503da49-9e18-4805-87da-490ea52a5a90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:44.222825 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0503da49-9e18-4805-87da-490ea52a5a90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:44.306004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0503da49-9e18-4805-87da-490ea52a5a90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:44.317637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0503da49-9e18-4805-87da-490ea52a5a90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.121112 Aug 07 01:17:44.318528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 374/773] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:44 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:44.948350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6fc2a358-f70f-4953-b930-7f973937ef62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:44.952380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6fc2a358-f70f-4953-b930-7f973937ef62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:44.960273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6fc2a358-f70f-4953-b930-7f973937ef62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:44.960775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6fc2a358-f70f-4953-b930-7f973937ef62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:44.961227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6fc2a358-f70f-4953-b930-7f973937ef62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:44.961970 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6fc2a358-f70f-4953-b930-7f973937ef62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:44.962431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6fc2a358-f70f-4953-b930-7f973937ef62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:44.962868 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6fc2a358-f70f-4953-b930-7f973937ef62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:45.010780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6fc2a358-f70f-4953-b930-7f973937ef62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069789 Aug 07 01:17:45.011862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 400/774] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:44 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:45.345198 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b2c3222c-1c9d-4b4b-ae9c-e687ea77d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:45.346091 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b2c3222c-1c9d-4b4b-ae9c-e687ea77d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:45.353949 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b2c3222c-1c9d-4b4b-ae9c-e687ea77d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:45.354447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b2c3222c-1c9d-4b4b-ae9c-e687ea77d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:45.354888 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b2c3222c-1c9d-4b4b-ae9c-e687ea77d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:45.355619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b2c3222c-1c9d-4b4b-ae9c-e687ea77d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:45.356080 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b2c3222c-1c9d-4b4b-ae9c-e687ea77d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:45.356849 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b2c3222c-1c9d-4b4b-ae9c-e687ea77d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:45.456595 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b2c3222c-1c9d-4b4b-ae9c-e687ea77d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:45.472060 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b2c3222c-1c9d-4b4b-ae9c-e687ea77d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.131229 Aug 07 01:17:45.473172 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 375/775] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:45 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:46.029154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5bbcef7f-c3fc-41bc-b389-07d6169910c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:46.032387 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5bbcef7f-c3fc-41bc-b389-07d6169910c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:46.039469 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bbcef7f-c3fc-41bc-b389-07d6169910c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:46.040159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bbcef7f-c3fc-41bc-b389-07d6169910c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:46.040696 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bbcef7f-c3fc-41bc-b389-07d6169910c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:46.041484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bbcef7f-c3fc-41bc-b389-07d6169910c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:46.042034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bbcef7f-c3fc-41bc-b389-07d6169910c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:46.042511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5bbcef7f-c3fc-41bc-b389-07d6169910c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:46.083023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5bbcef7f-c3fc-41bc-b389-07d6169910c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058393 Aug 07 01:17:46.083628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 401/776] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:46 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:46.507388 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7f6ada85-3a49-4fa5-85d3-b595cb4e3be4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:46.508416 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7f6ada85-3a49-4fa5-85d3-b595cb4e3be4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:46.516203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7f6ada85-3a49-4fa5-85d3-b595cb4e3be4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:46.516813 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7f6ada85-3a49-4fa5-85d3-b595cb4e3be4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:46.517330 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7f6ada85-3a49-4fa5-85d3-b595cb4e3be4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:46.518122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7f6ada85-3a49-4fa5-85d3-b595cb4e3be4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:46.518667 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7f6ada85-3a49-4fa5-85d3-b595cb4e3be4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:46.519148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7f6ada85-3a49-4fa5-85d3-b595cb4e3be4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:46.609447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7f6ada85-3a49-4fa5-85d3-b595cb4e3be4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:46.629810 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7f6ada85-3a49-4fa5-85d3-b595cb4e3be4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.128749 Aug 07 01:17:46.630896 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 376/777] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:46 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:47.104047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a0c71886-2a02-4d36-bba1-f6db4c402214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:47.108171 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a0c71886-2a02-4d36-bba1-f6db4c402214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:47.120760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0c71886-2a02-4d36-bba1-f6db4c402214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:47.121620 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0c71886-2a02-4d36-bba1-f6db4c402214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:47.122434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0c71886-2a02-4d36-bba1-f6db4c402214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:47.123777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0c71886-2a02-4d36-bba1-f6db4c402214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:47.124587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0c71886-2a02-4d36-bba1-f6db4c402214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:47.125289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0c71886-2a02-4d36-bba1-f6db4c402214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:47.185810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a0c71886-2a02-4d36-bba1-f6db4c402214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.089207 Aug 07 01:17:47.186918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 402/778] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:47 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:47.662060 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-67fd722b-7b92-4df0-a744-2b94d1028f21 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:47.663169 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-67fd722b-7b92-4df0-a744-2b94d1028f21 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:47.675508 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67fd722b-7b92-4df0-a744-2b94d1028f21 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:47.676314 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67fd722b-7b92-4df0-a744-2b94d1028f21 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:47.676952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67fd722b-7b92-4df0-a744-2b94d1028f21 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:47.678009 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67fd722b-7b92-4df0-a744-2b94d1028f21 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:47.678707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67fd722b-7b92-4df0-a744-2b94d1028f21 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:47.679439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67fd722b-7b92-4df0-a744-2b94d1028f21 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:47.755405 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-67fd722b-7b92-4df0-a744-2b94d1028f21 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:47.763941 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-67fd722b-7b92-4df0-a744-2b94d1028f21 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.109696 Aug 07 01:17:47.764763 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 377/779] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:47 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:48.206931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0e072dc5-e342-4ca9-baa5-f21b5f8642dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:48.210752 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0e072dc5-e342-4ca9-baa5-f21b5f8642dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:48.221381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e072dc5-e342-4ca9-baa5-f21b5f8642dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:48.221889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e072dc5-e342-4ca9-baa5-f21b5f8642dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:48.222340 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e072dc5-e342-4ca9-baa5-f21b5f8642dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:48.223519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e072dc5-e342-4ca9-baa5-f21b5f8642dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:48.224384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e072dc5-e342-4ca9-baa5-f21b5f8642dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:48.224904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e072dc5-e342-4ca9-baa5-f21b5f8642dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:48.260703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0e072dc5-e342-4ca9-baa5-f21b5f8642dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064268 Aug 07 01:17:48.261536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 403/780] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:48 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:48.786433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d168f42d-49a7-4474-93b0-e39610524f9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:48.787216 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d168f42d-49a7-4474-93b0-e39610524f9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:48.795743 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d168f42d-49a7-4474-93b0-e39610524f9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:48.796309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d168f42d-49a7-4474-93b0-e39610524f9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:48.796789 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d168f42d-49a7-4474-93b0-e39610524f9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:48.797552 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d168f42d-49a7-4474-93b0-e39610524f9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:48.798011 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d168f42d-49a7-4474-93b0-e39610524f9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:48.798603 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d168f42d-49a7-4474-93b0-e39610524f9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:48.871856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d168f42d-49a7-4474-93b0-e39610524f9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:48.880774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d168f42d-49a7-4474-93b0-e39610524f9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.098573 Aug 07 01:17:48.881369 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 378/781] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:48 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:49.275322 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fd0735f6-bbaf-4afd-962d-1e7477ecb176 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:49.277951 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fd0735f6-bbaf-4afd-962d-1e7477ecb176 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:49.285733 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd0735f6-bbaf-4afd-962d-1e7477ecb176 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:49.286259 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd0735f6-bbaf-4afd-962d-1e7477ecb176 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:49.286723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd0735f6-bbaf-4afd-962d-1e7477ecb176 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:49.287591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd0735f6-bbaf-4afd-962d-1e7477ecb176 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:49.288072 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd0735f6-bbaf-4afd-962d-1e7477ecb176 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:49.288559 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd0735f6-bbaf-4afd-962d-1e7477ecb176 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:49.325567 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fd0735f6-bbaf-4afd-962d-1e7477ecb176 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054324 Aug 07 01:17:49.326246 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 404/782] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:49 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:49.915976 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3628f4de-1d6b-43ee-ae1f-294e6c7763cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:49.916848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3628f4de-1d6b-43ee-ae1f-294e6c7763cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:49.924386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3628f4de-1d6b-43ee-ae1f-294e6c7763cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:49.924901 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3628f4de-1d6b-43ee-ae1f-294e6c7763cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:49.925409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3628f4de-1d6b-43ee-ae1f-294e6c7763cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:49.926399 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3628f4de-1d6b-43ee-ae1f-294e6c7763cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:49.926974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3628f4de-1d6b-43ee-ae1f-294e6c7763cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:49.927621 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3628f4de-1d6b-43ee-ae1f-294e6c7763cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:50.032429 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3628f4de-1d6b-43ee-ae1f-294e6c7763cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:50.042853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3628f4de-1d6b-43ee-ae1f-294e6c7763cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.131188 Aug 07 01:17:50.043601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 379/783] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:49 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:50.342058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4231b725-9368-460c-97ac-ba5daf647431 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:50.346188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4231b725-9368-460c-97ac-ba5daf647431 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:50.356478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4231b725-9368-460c-97ac-ba5daf647431 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:50.357212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4231b725-9368-460c-97ac-ba5daf647431 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:50.357795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4231b725-9368-460c-97ac-ba5daf647431 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:50.359529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4231b725-9368-460c-97ac-ba5daf647431 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:50.360265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4231b725-9368-460c-97ac-ba5daf647431 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:50.361038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4231b725-9368-460c-97ac-ba5daf647431 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:50.399195 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4231b725-9368-460c-97ac-ba5daf647431 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063270 Aug 07 01:17:50.399862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 405/784] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:50 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:51.079480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a955cb3a-b771-4119-9e1e-155a59ffa76b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:51.080226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a955cb3a-b771-4119-9e1e-155a59ffa76b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:51.088929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a955cb3a-b771-4119-9e1e-155a59ffa76b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:51.089483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a955cb3a-b771-4119-9e1e-155a59ffa76b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:51.090058 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a955cb3a-b771-4119-9e1e-155a59ffa76b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:51.090851 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a955cb3a-b771-4119-9e1e-155a59ffa76b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:51.091410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a955cb3a-b771-4119-9e1e-155a59ffa76b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:51.092033 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a955cb3a-b771-4119-9e1e-155a59ffa76b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:51.170477 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a955cb3a-b771-4119-9e1e-155a59ffa76b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:51.183383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a955cb3a-b771-4119-9e1e-155a59ffa76b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.108514 Aug 07 01:17:51.184433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 380/785] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:51 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:17:51.413745 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e18c1bef-160a-4dee-9584-d299d9bbf89d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:51.416002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e18c1bef-160a-4dee-9584-d299d9bbf89d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:51.425257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e18c1bef-160a-4dee-9584-d299d9bbf89d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:51.425920 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e18c1bef-160a-4dee-9584-d299d9bbf89d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:51.426405 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e18c1bef-160a-4dee-9584-d299d9bbf89d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:51.427178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e18c1bef-160a-4dee-9584-d299d9bbf89d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:51.427681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e18c1bef-160a-4dee-9584-d299d9bbf89d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:51.428153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e18c1bef-160a-4dee-9584-d299d9bbf89d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:51.464129 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e18c1bef-160a-4dee-9584-d299d9bbf89d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054939 Aug 07 01:17:51.464824 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 406/786] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:51 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:52.211263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f8ed9a6e-3969-4d67-910f-04ca5f0b120e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:52.212453 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f8ed9a6e-3969-4d67-910f-04ca5f0b120e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:52.224683 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8ed9a6e-3969-4d67-910f-04ca5f0b120e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:52.225548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8ed9a6e-3969-4d67-910f-04ca5f0b120e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:52.226342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8ed9a6e-3969-4d67-910f-04ca5f0b120e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:52.227658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8ed9a6e-3969-4d67-910f-04ca5f0b120e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:52.228955 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8ed9a6e-3969-4d67-910f-04ca5f0b120e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:52.229341 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8ed9a6e-3969-4d67-910f-04ca5f0b120e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:52.358801 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f8ed9a6e-3969-4d67-910f-04ca5f0b120e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:52.373168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f8ed9a6e-3969-4d67-910f-04ca5f0b120e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.169779 Aug 07 01:17:52.374255 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 381/787] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:52 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:52.478886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4cca4f80-0cdc-4b0b-b589-f5e067c75fdf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:52.481758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4cca4f80-0cdc-4b0b-b589-f5e067c75fdf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:52.490677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4cca4f80-0cdc-4b0b-b589-f5e067c75fdf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:52.491349 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4cca4f80-0cdc-4b0b-b589-f5e067c75fdf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:52.491884 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4cca4f80-0cdc-4b0b-b589-f5e067c75fdf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:52.492810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4cca4f80-0cdc-4b0b-b589-f5e067c75fdf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:52.493400 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4cca4f80-0cdc-4b0b-b589-f5e067c75fdf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:52.493928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4cca4f80-0cdc-4b0b-b589-f5e067c75fdf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:52.541639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4cca4f80-0cdc-4b0b-b589-f5e067c75fdf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066717 Aug 07 01:17:52.542726 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 407/788] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:52 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:53.410414 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-19fa3939-9b5d-43f6-8886-da9d6f37609e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:53.411555 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-19fa3939-9b5d-43f6-8886-da9d6f37609e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:53.423969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19fa3939-9b5d-43f6-8886-da9d6f37609e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:53.424914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19fa3939-9b5d-43f6-8886-da9d6f37609e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:53.425634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19fa3939-9b5d-43f6-8886-da9d6f37609e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:53.426848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19fa3939-9b5d-43f6-8886-da9d6f37609e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:53.427607 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19fa3939-9b5d-43f6-8886-da9d6f37609e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:53.428482 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19fa3939-9b5d-43f6-8886-da9d6f37609e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:53.531459 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-19fa3939-9b5d-43f6-8886-da9d6f37609e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:53.541889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-19fa3939-9b5d-43f6-8886-da9d6f37609e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.139323 Aug 07 01:17:53.542638 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 382/789] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:53 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:53.565235 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2ec85de0-9e46-45e8-acea-6d196d843b28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:53.569312 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2ec85de0-9e46-45e8-acea-6d196d843b28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:53.583095 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2ec85de0-9e46-45e8-acea-6d196d843b28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:53.583977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2ec85de0-9e46-45e8-acea-6d196d843b28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:53.585086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2ec85de0-9e46-45e8-acea-6d196d843b28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:53.586522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2ec85de0-9e46-45e8-acea-6d196d843b28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:53.587366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2ec85de0-9e46-45e8-acea-6d196d843b28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:53.588161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2ec85de0-9e46-45e8-acea-6d196d843b28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:53.653353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2ec85de0-9e46-45e8-acea-6d196d843b28 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.097499 Aug 07 01:17:53.654478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 408/790] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:53 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:54.570400 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-15070723-d652-439d-9f5a-d87b07767235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:54.571323 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-15070723-d652-439d-9f5a-d87b07767235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:54.583838 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15070723-d652-439d-9f5a-d87b07767235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:54.584700 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15070723-d652-439d-9f5a-d87b07767235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:54.585448 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15070723-d652-439d-9f5a-d87b07767235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:54.586712 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15070723-d652-439d-9f5a-d87b07767235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:54.587504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15070723-d652-439d-9f5a-d87b07767235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:54.588324 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15070723-d652-439d-9f5a-d87b07767235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:54.669883 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e8d35e52-1353-46a1-b75d-8ef27d24a559 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:54.672414 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e8d35e52-1353-46a1-b75d-8ef27d24a559 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:54.680271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e8d35e52-1353-46a1-b75d-8ef27d24a559 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:54.680766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e8d35e52-1353-46a1-b75d-8ef27d24a559 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:54.681232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e8d35e52-1353-46a1-b75d-8ef27d24a559 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:54.681447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-15070723-d652-439d-9f5a-d87b07767235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:54.681956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e8d35e52-1353-46a1-b75d-8ef27d24a559 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:54.682419 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e8d35e52-1353-46a1-b75d-8ef27d24a559 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:54.682842 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e8d35e52-1353-46a1-b75d-8ef27d24a559 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:54.690929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-15070723-d652-439d-9f5a-d87b07767235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.128029 Aug 07 01:17:54.691529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 383/791] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:54 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:54.720423 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e8d35e52-1353-46a1-b75d-8ef27d24a559 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054656 Aug 07 01:17:54.721040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 409/792] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:54 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:55.726165 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-008beeb3-38ee-4e9f-82b2-3aee9bdfa2a9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:55.727027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-008beeb3-38ee-4e9f-82b2-3aee9bdfa2a9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:55.733773 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e17025ae-827e-4b99-bafe-a0b5f52388d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:55.734664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-008beeb3-38ee-4e9f-82b2-3aee9bdfa2a9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:55.735074 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-008beeb3-38ee-4e9f-82b2-3aee9bdfa2a9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:55.735510 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-008beeb3-38ee-4e9f-82b2-3aee9bdfa2a9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:55.736409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-008beeb3-38ee-4e9f-82b2-3aee9bdfa2a9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:55.736601 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e17025ae-827e-4b99-bafe-a0b5f52388d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:55.736871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-008beeb3-38ee-4e9f-82b2-3aee9bdfa2a9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:55.737299 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-008beeb3-38ee-4e9f-82b2-3aee9bdfa2a9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:55.744631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e17025ae-827e-4b99-bafe-a0b5f52388d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:55.745265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e17025ae-827e-4b99-bafe-a0b5f52388d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:55.745837 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e17025ae-827e-4b99-bafe-a0b5f52388d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:55.746707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e17025ae-827e-4b99-bafe-a0b5f52388d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:55.747265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e17025ae-827e-4b99-bafe-a0b5f52388d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:55.747840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e17025ae-827e-4b99-bafe-a0b5f52388d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:55.784003 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e17025ae-827e-4b99-bafe-a0b5f52388d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054835 Aug 07 01:17:55.784770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 410/793] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:55 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:17:55.815159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-008beeb3-38ee-4e9f-82b2-3aee9bdfa2a9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:55.824141 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-008beeb3-38ee-4e9f-82b2-3aee9bdfa2a9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.101894 Aug 07 01:17:55.824792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 384/794] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:55 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:56.803273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fd2c8914-bbf1-4ca6-9254-9ea417d4489b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:56.807184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fd2c8914-bbf1-4ca6-9254-9ea417d4489b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:56.819718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd2c8914-bbf1-4ca6-9254-9ea417d4489b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:56.820635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd2c8914-bbf1-4ca6-9254-9ea417d4489b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:56.821463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd2c8914-bbf1-4ca6-9254-9ea417d4489b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:56.822723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd2c8914-bbf1-4ca6-9254-9ea417d4489b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:56.823553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd2c8914-bbf1-4ca6-9254-9ea417d4489b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:56.824356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd2c8914-bbf1-4ca6-9254-9ea417d4489b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:56.853608 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-81bc34c2-9825-478f-b6fe-9fc1bddb4b64 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:56.854828 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-81bc34c2-9825-478f-b6fe-9fc1bddb4b64 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:56.868750 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-81bc34c2-9825-478f-b6fe-9fc1bddb4b64 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:56.869636 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-81bc34c2-9825-478f-b6fe-9fc1bddb4b64 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:56.870498 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-81bc34c2-9825-478f-b6fe-9fc1bddb4b64 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:56.872068 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-81bc34c2-9825-478f-b6fe-9fc1bddb4b64 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:56.873208 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-81bc34c2-9825-478f-b6fe-9fc1bddb4b64 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:56.874338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-81bc34c2-9825-478f-b6fe-9fc1bddb4b64 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:56.883607 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fd2c8914-bbf1-4ca6-9254-9ea417d4489b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.087674 Aug 07 01:17:56.884582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 411/795] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:56 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:57.003723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-81bc34c2-9825-478f-b6fe-9fc1bddb4b64 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:57.014526 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-81bc34c2-9825-478f-b6fe-9fc1bddb4b64 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.170034 Aug 07 01:17:57.015292 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 385/796] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:56 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:57.900590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e8435738-d256-4fee-b76a-f1aa10b8b545 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:57.903442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e8435738-d256-4fee-b76a-f1aa10b8b545 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:57.912439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e8435738-d256-4fee-b76a-f1aa10b8b545 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:57.912982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e8435738-d256-4fee-b76a-f1aa10b8b545 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:57.913462 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e8435738-d256-4fee-b76a-f1aa10b8b545 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:57.914224 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e8435738-d256-4fee-b76a-f1aa10b8b545 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:57.914829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e8435738-d256-4fee-b76a-f1aa10b8b545 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:57.915364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e8435738-d256-4fee-b76a-f1aa10b8b545 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:57.957079 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e8435738-d256-4fee-b76a-f1aa10b8b545 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061766 Aug 07 01:17:57.957786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 412/797] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:57 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:58.050254 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-950f4cef-8932-4d10-83fd-226779014ea0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:58.050934 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-950f4cef-8932-4d10-83fd-226779014ea0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:58.059012 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-950f4cef-8932-4d10-83fd-226779014ea0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:58.059547 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-950f4cef-8932-4d10-83fd-226779014ea0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:58.060009 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-950f4cef-8932-4d10-83fd-226779014ea0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:58.061543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-950f4cef-8932-4d10-83fd-226779014ea0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:58.062176 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-950f4cef-8932-4d10-83fd-226779014ea0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:58.062679 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-950f4cef-8932-4d10-83fd-226779014ea0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:58.158802 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-950f4cef-8932-4d10-83fd-226779014ea0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:58.168013 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-950f4cef-8932-4d10-83fd-226779014ea0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.123120 Aug 07 01:17:58.168886 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 386/798] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:58 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:17:58.974550 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-635caf0f-b34f-46f8-8b54-c73a4cf2b029 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:58.977715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-635caf0f-b34f-46f8-8b54-c73a4cf2b029 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:58.986728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-635caf0f-b34f-46f8-8b54-c73a4cf2b029 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:58.987264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-635caf0f-b34f-46f8-8b54-c73a4cf2b029 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:58.987726 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-635caf0f-b34f-46f8-8b54-c73a4cf2b029 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:58.988820 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-635caf0f-b34f-46f8-8b54-c73a4cf2b029 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:58.989414 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-635caf0f-b34f-46f8-8b54-c73a4cf2b029 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:58.989893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-635caf0f-b34f-46f8-8b54-c73a4cf2b029 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:59.023674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:17:59.031959 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-635caf0f-b34f-46f8-8b54-c73a4cf2b029 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061446 Aug 07 01:17:59.032648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 413/799] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:17:58 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:17:59.198191 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f5ae4ddd-b852-4e27-b313-4374b71182d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:17:59.198832 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f5ae4ddd-b852-4e27-b313-4374b71182d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:17:59.210105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5ae4ddd-b852-4e27-b313-4374b71182d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:59.210644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5ae4ddd-b852-4e27-b313-4374b71182d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:59.211093 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5ae4ddd-b852-4e27-b313-4374b71182d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:59.211843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5ae4ddd-b852-4e27-b313-4374b71182d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:17:59.212349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5ae4ddd-b852-4e27-b313-4374b71182d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:17:59.212782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5ae4ddd-b852-4e27-b313-4374b71182d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:17:59.307139 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f5ae4ddd-b852-4e27-b313-4374b71182d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:17:59.317293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f5ae4ddd-b852-4e27-b313-4374b71182d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.125028 Aug 07 01:17:59.318159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 387/800] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:17:59 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:00.047878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a7fc9fe7-0905-4bff-8c12-d1dbbcb1ccbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:00.051146 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a7fc9fe7-0905-4bff-8c12-d1dbbcb1ccbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:00.054886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=56 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:18:00.065164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7fc9fe7-0905-4bff-8c12-d1dbbcb1ccbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:00.065164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7fc9fe7-0905-4bff-8c12-d1dbbcb1ccbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:00.065164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7fc9fe7-0905-4bff-8c12-d1dbbcb1ccbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:00.067740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7fc9fe7-0905-4bff-8c12-d1dbbcb1ccbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:00.067740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7fc9fe7-0905-4bff-8c12-d1dbbcb1ccbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:00.067740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7fc9fe7-0905-4bff-8c12-d1dbbcb1ccbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:00.123975 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a7fc9fe7-0905-4bff-8c12-d1dbbcb1ccbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.082460 Aug 07 01:18:00.125277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 414/801] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:00 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:18:00.344728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-de3c4d08-f1e3-4eb3-a4bf-377fec71548c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:00.345338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-de3c4d08-f1e3-4eb3-a4bf-377fec71548c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:00.347746 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=54 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:18:00.353819 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-de3c4d08-f1e3-4eb3-a4bf-377fec71548c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:00.354320 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-de3c4d08-f1e3-4eb3-a4bf-377fec71548c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:00.354767 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-de3c4d08-f1e3-4eb3-a4bf-377fec71548c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:00.355490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-de3c4d08-f1e3-4eb3-a4bf-377fec71548c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:00.355951 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-de3c4d08-f1e3-4eb3-a4bf-377fec71548c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:00.356590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-de3c4d08-f1e3-4eb3-a4bf-377fec71548c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:00.446176 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-de3c4d08-f1e3-4eb3-a4bf-377fec71548c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:00.463194 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-de3c4d08-f1e3-4eb3-a4bf-377fec71548c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.123155 Aug 07 01:18:00.463847 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 388/802] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:00 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:01.139851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7ad97e19-5c1f-4d24-ab52-92b915d18ae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:01.143680 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7ad97e19-5c1f-4d24-ab52-92b915d18ae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:01.153181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ad97e19-5c1f-4d24-ab52-92b915d18ae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:01.153755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ad97e19-5c1f-4d24-ab52-92b915d18ae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:01.154210 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ad97e19-5c1f-4d24-ab52-92b915d18ae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:01.154974 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ad97e19-5c1f-4d24-ab52-92b915d18ae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:01.155445 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ad97e19-5c1f-4d24-ab52-92b915d18ae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:01.155881 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ad97e19-5c1f-4d24-ab52-92b915d18ae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:01.195252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7ad97e19-5c1f-4d24-ab52-92b915d18ae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060579 Aug 07 01:18:01.196585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 415/803] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:01 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:01.488042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f5a6d2dc-2dc4-4f27-ad64-aceea69c93de tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:01.488863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f5a6d2dc-2dc4-4f27-ad64-aceea69c93de tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:01.496097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5a6d2dc-2dc4-4f27-ad64-aceea69c93de tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:01.496775 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5a6d2dc-2dc4-4f27-ad64-aceea69c93de tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:01.497328 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5a6d2dc-2dc4-4f27-ad64-aceea69c93de tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:01.498752 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5a6d2dc-2dc4-4f27-ad64-aceea69c93de tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:01.499339 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5a6d2dc-2dc4-4f27-ad64-aceea69c93de tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:01.499885 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5a6d2dc-2dc4-4f27-ad64-aceea69c93de tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:01.504842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:18:01.587845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f5a6d2dc-2dc4-4f27-ad64-aceea69c93de tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:01.601688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f5a6d2dc-2dc4-4f27-ad64-aceea69c93de tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.117423 Aug 07 01:18:01.602971 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 389/804] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:01 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:02.209769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5b8433e7-8b09-432b-b402-c61a585de20c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:02.212525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5b8433e7-8b09-432b-b402-c61a585de20c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:02.220802 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b8433e7-8b09-432b-b402-c61a585de20c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:02.221426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b8433e7-8b09-432b-b402-c61a585de20c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:02.222016 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b8433e7-8b09-432b-b402-c61a585de20c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:02.222932 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b8433e7-8b09-432b-b402-c61a585de20c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:02.223454 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b8433e7-8b09-432b-b402-c61a585de20c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:02.223937 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b8433e7-8b09-432b-b402-c61a585de20c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:02.266405 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5b8433e7-8b09-432b-b402-c61a585de20c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060859 Aug 07 01:18:02.267021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 416/805] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:02 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:02.637016 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3247c263-a9e2-41a7-a70f-4263f6a0b5f4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:02.637905 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3247c263-a9e2-41a7-a70f-4263f6a0b5f4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:02.645739 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3247c263-a9e2-41a7-a70f-4263f6a0b5f4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:02.646295 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3247c263-a9e2-41a7-a70f-4263f6a0b5f4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:02.646781 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3247c263-a9e2-41a7-a70f-4263f6a0b5f4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:02.647615 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3247c263-a9e2-41a7-a70f-4263f6a0b5f4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:02.648335 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3247c263-a9e2-41a7-a70f-4263f6a0b5f4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:02.648876 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3247c263-a9e2-41a7-a70f-4263f6a0b5f4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:02.727324 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3247c263-a9e2-41a7-a70f-4263f6a0b5f4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:02.736563 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3247c263-a9e2-41a7-a70f-4263f6a0b5f4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.103811 Aug 07 01:18:02.737180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 390/806] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:02 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:03.283733 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-00c19c5a-8152-4b05-904d-61465f75af0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:03.286062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-00c19c5a-8152-4b05-904d-61465f75af0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:03.297124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-00c19c5a-8152-4b05-904d-61465f75af0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:03.297698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-00c19c5a-8152-4b05-904d-61465f75af0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:03.298148 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-00c19c5a-8152-4b05-904d-61465f75af0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:03.298931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-00c19c5a-8152-4b05-904d-61465f75af0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:03.299400 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-00c19c5a-8152-4b05-904d-61465f75af0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:03.299960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-00c19c5a-8152-4b05-904d-61465f75af0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:03.339269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-00c19c5a-8152-4b05-904d-61465f75af0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063461 Aug 07 01:18:03.340001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 417/807] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:03 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:18:03.772692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-10381345-9b48-4f40-add1-ee0abe185ff7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:03.773545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-10381345-9b48-4f40-add1-ee0abe185ff7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:03.782104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10381345-9b48-4f40-add1-ee0abe185ff7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:03.782743 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10381345-9b48-4f40-add1-ee0abe185ff7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:03.783466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10381345-9b48-4f40-add1-ee0abe185ff7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:03.784345 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10381345-9b48-4f40-add1-ee0abe185ff7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:03.784953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10381345-9b48-4f40-add1-ee0abe185ff7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:03.785523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10381345-9b48-4f40-add1-ee0abe185ff7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:03.864324 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-10381345-9b48-4f40-add1-ee0abe185ff7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:03.878009 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-10381345-9b48-4f40-add1-ee0abe185ff7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.109275 Aug 07 01:18:03.878708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 391/808] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:03 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:04.356296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c257c199-6bef-45b2-8d98-bf699969f011 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:04.359040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c257c199-6bef-45b2-8d98-bf699969f011 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:04.366423 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c257c199-6bef-45b2-8d98-bf699969f011 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:04.366916 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c257c199-6bef-45b2-8d98-bf699969f011 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:04.367368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c257c199-6bef-45b2-8d98-bf699969f011 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:04.368130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c257c199-6bef-45b2-8d98-bf699969f011 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:04.368614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c257c199-6bef-45b2-8d98-bf699969f011 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:04.369042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c257c199-6bef-45b2-8d98-bf699969f011 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:04.410994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c257c199-6bef-45b2-8d98-bf699969f011 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061211 Aug 07 01:18:04.411585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 418/809] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:04 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:04.912100 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e3abebe6-9c95-4fd0-a85e-b41557252dab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:04.913062 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e3abebe6-9c95-4fd0-a85e-b41557252dab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:04.920974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3abebe6-9c95-4fd0-a85e-b41557252dab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:04.921480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3abebe6-9c95-4fd0-a85e-b41557252dab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:04.921953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3abebe6-9c95-4fd0-a85e-b41557252dab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:04.922735 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3abebe6-9c95-4fd0-a85e-b41557252dab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:04.923199 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3abebe6-9c95-4fd0-a85e-b41557252dab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:04.923649 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3abebe6-9c95-4fd0-a85e-b41557252dab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:05.028748 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e3abebe6-9c95-4fd0-a85e-b41557252dab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:05.042726 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e3abebe6-9c95-4fd0-a85e-b41557252dab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.134323 Aug 07 01:18:05.043641 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 392/810] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:04 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:05.424904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e49378c1-3f72-4eef-833a-a1b5fb8c57ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:05.427194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e49378c1-3f72-4eef-833a-a1b5fb8c57ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:05.434107 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e49378c1-3f72-4eef-833a-a1b5fb8c57ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:05.434577 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e49378c1-3f72-4eef-833a-a1b5fb8c57ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:05.435017 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e49378c1-3f72-4eef-833a-a1b5fb8c57ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:05.435848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e49378c1-3f72-4eef-833a-a1b5fb8c57ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:05.436453 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e49378c1-3f72-4eef-833a-a1b5fb8c57ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:05.436968 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e49378c1-3f72-4eef-833a-a1b5fb8c57ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:05.473474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e49378c1-3f72-4eef-833a-a1b5fb8c57ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053446 Aug 07 01:18:05.474067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 419/811] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:05 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:06.080346 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-efe8f383-fb0f-4131-8bde-6e5c19bd7f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:06.081219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-efe8f383-fb0f-4131-8bde-6e5c19bd7f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:06.089773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-efe8f383-fb0f-4131-8bde-6e5c19bd7f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:06.090359 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-efe8f383-fb0f-4131-8bde-6e5c19bd7f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:06.090887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-efe8f383-fb0f-4131-8bde-6e5c19bd7f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:06.091725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-efe8f383-fb0f-4131-8bde-6e5c19bd7f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:06.092943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-efe8f383-fb0f-4131-8bde-6e5c19bd7f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:06.093517 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-efe8f383-fb0f-4131-8bde-6e5c19bd7f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:06.166312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-efe8f383-fb0f-4131-8bde-6e5c19bd7f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:06.175960 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-efe8f383-fb0f-4131-8bde-6e5c19bd7f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100430 Aug 07 01:18:06.176609 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 393/812] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:06 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:06.487634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3c5d8f4e-f84c-4daf-8708-d2657415c6e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:06.492442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3c5d8f4e-f84c-4daf-8708-d2657415c6e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:06.498425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c5d8f4e-f84c-4daf-8708-d2657415c6e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:06.498928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c5d8f4e-f84c-4daf-8708-d2657415c6e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:06.499473 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c5d8f4e-f84c-4daf-8708-d2657415c6e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:06.500223 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c5d8f4e-f84c-4daf-8708-d2657415c6e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:06.500730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c5d8f4e-f84c-4daf-8708-d2657415c6e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:06.501170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c5d8f4e-f84c-4daf-8708-d2657415c6e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:06.536988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3c5d8f4e-f84c-4daf-8708-d2657415c6e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054378 Aug 07 01:18:06.537751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 420/813] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:06 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:18:07.211706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-77046f56-e37d-4965-878a-c0d59db2b284 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:07.212466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-77046f56-e37d-4965-878a-c0d59db2b284 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:07.219912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77046f56-e37d-4965-878a-c0d59db2b284 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:07.220593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77046f56-e37d-4965-878a-c0d59db2b284 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:07.220893 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77046f56-e37d-4965-878a-c0d59db2b284 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:07.221623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77046f56-e37d-4965-878a-c0d59db2b284 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:07.222087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77046f56-e37d-4965-878a-c0d59db2b284 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:07.222520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77046f56-e37d-4965-878a-c0d59db2b284 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:07.296810 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-77046f56-e37d-4965-878a-c0d59db2b284 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:07.308635 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-77046f56-e37d-4965-878a-c0d59db2b284 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.101012 Aug 07 01:18:07.309290 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 394/814] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:07 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:07.550933 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-63368745-930a-49b1-8a77-f790ba2b64a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:07.553501 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-63368745-930a-49b1-8a77-f790ba2b64a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:07.560762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63368745-930a-49b1-8a77-f790ba2b64a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:07.561252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63368745-930a-49b1-8a77-f790ba2b64a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:07.561694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63368745-930a-49b1-8a77-f790ba2b64a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:07.562526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63368745-930a-49b1-8a77-f790ba2b64a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:07.562995 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63368745-930a-49b1-8a77-f790ba2b64a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:07.563534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63368745-930a-49b1-8a77-f790ba2b64a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:07.602700 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-63368745-930a-49b1-8a77-f790ba2b64a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055810 Aug 07 01:18:07.603593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 421/815] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:07 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:08.333579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c4eec774-f957-48c9-81f3-0c35bde5571f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:08.334655 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c4eec774-f957-48c9-81f3-0c35bde5571f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:08.343166 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4eec774-f957-48c9-81f3-0c35bde5571f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:08.343669 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4eec774-f957-48c9-81f3-0c35bde5571f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:08.344114 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4eec774-f957-48c9-81f3-0c35bde5571f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:08.344898 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4eec774-f957-48c9-81f3-0c35bde5571f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:08.345365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4eec774-f957-48c9-81f3-0c35bde5571f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:08.345828 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c4eec774-f957-48c9-81f3-0c35bde5571f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:08.430502 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c4eec774-f957-48c9-81f3-0c35bde5571f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:08.440670 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c4eec774-f957-48c9-81f3-0c35bde5571f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.114886 Aug 07 01:18:08.441454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 395/816] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:08 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:08.616890 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-279eb6e9-d426-45f1-90e5-5ad48c84ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:08.619012 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-279eb6e9-d426-45f1-90e5-5ad48c84ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:08.627142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-279eb6e9-d426-45f1-90e5-5ad48c84ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:08.627638 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-279eb6e9-d426-45f1-90e5-5ad48c84ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:08.628083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-279eb6e9-d426-45f1-90e5-5ad48c84ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:08.628847 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-279eb6e9-d426-45f1-90e5-5ad48c84ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:08.629310 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-279eb6e9-d426-45f1-90e5-5ad48c84ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:08.629759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-279eb6e9-d426-45f1-90e5-5ad48c84ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:08.665267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-279eb6e9-d426-45f1-90e5-5ad48c84ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052866 Aug 07 01:18:08.665971 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 422/817] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:08 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:09.465949 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d315590f-3d66-422d-a694-ff1455a55ba5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:09.467428 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d315590f-3d66-422d-a694-ff1455a55ba5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:09.474461 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d315590f-3d66-422d-a694-ff1455a55ba5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:09.474960 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d315590f-3d66-422d-a694-ff1455a55ba5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:09.475406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d315590f-3d66-422d-a694-ff1455a55ba5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:09.476129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d315590f-3d66-422d-a694-ff1455a55ba5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:09.476634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d315590f-3d66-422d-a694-ff1455a55ba5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:09.477106 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d315590f-3d66-422d-a694-ff1455a55ba5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:09.569129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d315590f-3d66-422d-a694-ff1455a55ba5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:09.577971 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d315590f-3d66-422d-a694-ff1455a55ba5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.116414 Aug 07 01:18:09.578725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 396/818] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:09 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:09.679355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1c7bbc19-c1dc-48b2-b4ce-026926786ff5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:09.682154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1c7bbc19-c1dc-48b2-b4ce-026926786ff5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:09.689281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c7bbc19-c1dc-48b2-b4ce-026926786ff5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:09.689940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c7bbc19-c1dc-48b2-b4ce-026926786ff5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:09.690503 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c7bbc19-c1dc-48b2-b4ce-026926786ff5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:09.691359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c7bbc19-c1dc-48b2-b4ce-026926786ff5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:09.691928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c7bbc19-c1dc-48b2-b4ce-026926786ff5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:09.692519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c7bbc19-c1dc-48b2-b4ce-026926786ff5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:09.728705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1c7bbc19-c1dc-48b2-b4ce-026926786ff5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053416 Aug 07 01:18:09.729393 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 423/819] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:09 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:10.601572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0b604d71-129e-4900-9eff-fe170f83303b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:10.602169 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0b604d71-129e-4900-9eff-fe170f83303b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:10.610048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b604d71-129e-4900-9eff-fe170f83303b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:10.610847 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b604d71-129e-4900-9eff-fe170f83303b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:10.611466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b604d71-129e-4900-9eff-fe170f83303b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:10.612343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b604d71-129e-4900-9eff-fe170f83303b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:10.613021 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b604d71-129e-4900-9eff-fe170f83303b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:10.613548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b604d71-129e-4900-9eff-fe170f83303b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:10.686467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0b604d71-129e-4900-9eff-fe170f83303b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:10.695399 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0b604d71-129e-4900-9eff-fe170f83303b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097853 Aug 07 01:18:10.696064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 397/820] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:10 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:10.743310 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6112e078-055d-4e60-a95e-00e9ea647f18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:10.745694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6112e078-055d-4e60-a95e-00e9ea647f18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:10.753123 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6112e078-055d-4e60-a95e-00e9ea647f18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:10.753670 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6112e078-055d-4e60-a95e-00e9ea647f18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:10.754162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6112e078-055d-4e60-a95e-00e9ea647f18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:10.754942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6112e078-055d-4e60-a95e-00e9ea647f18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:10.755437 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6112e078-055d-4e60-a95e-00e9ea647f18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:10.755904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6112e078-055d-4e60-a95e-00e9ea647f18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:10.791598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6112e078-055d-4e60-a95e-00e9ea647f18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052384 Aug 07 01:18:10.792341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 424/821] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:10 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:11.718901 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-90750745-5016-470d-abaa-4d22da598c77 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:11.719811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-90750745-5016-470d-abaa-4d22da598c77 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:11.727047 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-90750745-5016-470d-abaa-4d22da598c77 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:11.727548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-90750745-5016-470d-abaa-4d22da598c77 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:11.727993 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-90750745-5016-470d-abaa-4d22da598c77 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:11.728790 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-90750745-5016-470d-abaa-4d22da598c77 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:11.729254 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-90750745-5016-470d-abaa-4d22da598c77 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:11.729835 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-90750745-5016-470d-abaa-4d22da598c77 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:11.803872 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-90750745-5016-470d-abaa-4d22da598c77 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:11.809156 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-88e03922-0ca3-4a0d-9e8e-01e8fc6d6df6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:11.813452 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-88e03922-0ca3-4a0d-9e8e-01e8fc6d6df6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:11.817784 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-90750745-5016-470d-abaa-4d22da598c77 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.102752 Aug 07 01:18:11.819411 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 398/822] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:11 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:11.824193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88e03922-0ca3-4a0d-9e8e-01e8fc6d6df6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:11.824910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88e03922-0ca3-4a0d-9e8e-01e8fc6d6df6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:11.825526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88e03922-0ca3-4a0d-9e8e-01e8fc6d6df6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:11.826970 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88e03922-0ca3-4a0d-9e8e-01e8fc6d6df6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:11.827878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88e03922-0ca3-4a0d-9e8e-01e8fc6d6df6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:11.828521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88e03922-0ca3-4a0d-9e8e-01e8fc6d6df6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:11.871986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-88e03922-0ca3-4a0d-9e8e-01e8fc6d6df6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.070542 Aug 07 01:18:11.873438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 425/823] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:11 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:12.847602 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9434039f-91ab-426c-bf48-df26319f399c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:12.848421 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9434039f-91ab-426c-bf48-df26319f399c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:12.858789 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9434039f-91ab-426c-bf48-df26319f399c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:12.859383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9434039f-91ab-426c-bf48-df26319f399c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:12.859865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9434039f-91ab-426c-bf48-df26319f399c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:12.860731 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9434039f-91ab-426c-bf48-df26319f399c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:12.861194 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9434039f-91ab-426c-bf48-df26319f399c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:12.861648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9434039f-91ab-426c-bf48-df26319f399c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:12.888707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0004687f-34a6-4fab-9d74-be7c4796c808 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:12.891769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0004687f-34a6-4fab-9d74-be7c4796c808 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:12.901286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0004687f-34a6-4fab-9d74-be7c4796c808 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:12.901789 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0004687f-34a6-4fab-9d74-be7c4796c808 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:12.902276 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0004687f-34a6-4fab-9d74-be7c4796c808 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:12.903022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0004687f-34a6-4fab-9d74-be7c4796c808 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:12.903490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0004687f-34a6-4fab-9d74-be7c4796c808 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:12.903998 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0004687f-34a6-4fab-9d74-be7c4796c808 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:12.950519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0004687f-34a6-4fab-9d74-be7c4796c808 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067444 Aug 07 01:18:12.951727 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 426/824] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:12 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:18:12.954464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9434039f-91ab-426c-bf48-df26319f399c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:12.968390 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9434039f-91ab-426c-bf48-df26319f399c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.125000 Aug 07 01:18:12.969416 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 399/825] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:12 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:13.965354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-00e89ec0-ddd8-485d-b465-82e99ac8d234 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:13.967738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-00e89ec0-ddd8-485d-b465-82e99ac8d234 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:13.975314 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-00e89ec0-ddd8-485d-b465-82e99ac8d234 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:13.975833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-00e89ec0-ddd8-485d-b465-82e99ac8d234 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:13.976306 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-00e89ec0-ddd8-485d-b465-82e99ac8d234 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:13.977032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-00e89ec0-ddd8-485d-b465-82e99ac8d234 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:13.977491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-00e89ec0-ddd8-485d-b465-82e99ac8d234 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:13.977947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-00e89ec0-ddd8-485d-b465-82e99ac8d234 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:14.006307 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2a5ac2f8-3280-4681-b66f-cb9345669452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:14.007433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2a5ac2f8-3280-4681-b66f-cb9345669452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:14.019411 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a5ac2f8-3280-4681-b66f-cb9345669452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:14.020325 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a5ac2f8-3280-4681-b66f-cb9345669452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:14.021084 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a5ac2f8-3280-4681-b66f-cb9345669452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:14.022375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a5ac2f8-3280-4681-b66f-cb9345669452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:14.023177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-00e89ec0-ddd8-485d-b465-82e99ac8d234 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061581 Aug 07 01:18:14.023302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a5ac2f8-3280-4681-b66f-cb9345669452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:14.023943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a5ac2f8-3280-4681-b66f-cb9345669452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:14.024306 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 427/826] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:13 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:18:14.118966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2a5ac2f8-3280-4681-b66f-cb9345669452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:14.128406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2a5ac2f8-3280-4681-b66f-cb9345669452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.128855 Aug 07 01:18:14.129317 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 400/827] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:13 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:15.037776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8a318b94-b166-40bc-976b-f4c1b69b0e81 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:15.041572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8a318b94-b166-40bc-976b-f4c1b69b0e81 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:15.052800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a318b94-b166-40bc-976b-f4c1b69b0e81 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:15.053756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a318b94-b166-40bc-976b-f4c1b69b0e81 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:15.054618 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a318b94-b166-40bc-976b-f4c1b69b0e81 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:15.055901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a318b94-b166-40bc-976b-f4c1b69b0e81 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:15.056695 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a318b94-b166-40bc-976b-f4c1b69b0e81 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:15.057448 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a318b94-b166-40bc-976b-f4c1b69b0e81 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:15.134271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8a318b94-b166-40bc-976b-f4c1b69b0e81 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.100647 Aug 07 01:18:15.135325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 428/828] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:15 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:15.156309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f8e55e14-0fb9-4761-972c-edd762358f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:15.158367 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f8e55e14-0fb9-4761-972c-edd762358f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:15.168481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8e55e14-0fb9-4761-972c-edd762358f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:15.169354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8e55e14-0fb9-4761-972c-edd762358f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:15.170149 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8e55e14-0fb9-4761-972c-edd762358f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:15.171449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8e55e14-0fb9-4761-972c-edd762358f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:15.172330 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8e55e14-0fb9-4761-972c-edd762358f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:15.173060 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8e55e14-0fb9-4761-972c-edd762358f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:15.291326 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f8e55e14-0fb9-4761-972c-edd762358f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:15.300204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f8e55e14-0fb9-4761-972c-edd762358f91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.152208 Aug 07 01:18:15.300855 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 401/829] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:15 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:16.157179 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-808f0c07-fd3a-4f9e-b220-4fcc44875083 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:16.160993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-808f0c07-fd3a-4f9e-b220-4fcc44875083 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:16.173519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-808f0c07-fd3a-4f9e-b220-4fcc44875083 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:16.174385 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-808f0c07-fd3a-4f9e-b220-4fcc44875083 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:16.175178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-808f0c07-fd3a-4f9e-b220-4fcc44875083 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:16.176503 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-808f0c07-fd3a-4f9e-b220-4fcc44875083 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:16.177362 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-808f0c07-fd3a-4f9e-b220-4fcc44875083 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:16.178184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-808f0c07-fd3a-4f9e-b220-4fcc44875083 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:16.245925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-808f0c07-fd3a-4f9e-b220-4fcc44875083 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.096578 Aug 07 01:18:16.246937 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 429/830] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:16 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:16.332707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1d170978-206b-4fbc-8f5c-1138ae8dda67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:16.333534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1d170978-206b-4fbc-8f5c-1138ae8dda67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:16.340724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d170978-206b-4fbc-8f5c-1138ae8dda67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:16.341772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d170978-206b-4fbc-8f5c-1138ae8dda67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:16.342217 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d170978-206b-4fbc-8f5c-1138ae8dda67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:16.343015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d170978-206b-4fbc-8f5c-1138ae8dda67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:16.343486 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d170978-206b-4fbc-8f5c-1138ae8dda67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:16.343917 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d170978-206b-4fbc-8f5c-1138ae8dda67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:16.420452 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1d170978-206b-4fbc-8f5c-1138ae8dda67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:16.430191 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1d170978-206b-4fbc-8f5c-1138ae8dda67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.101332 Aug 07 01:18:16.430937 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 402/831] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:16 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:17.263510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c6d79017-2b01-48f5-acd9-784ceffd89dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:17.266144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c6d79017-2b01-48f5-acd9-784ceffd89dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:17.273484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c6d79017-2b01-48f5-acd9-784ceffd89dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:17.274149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c6d79017-2b01-48f5-acd9-784ceffd89dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:17.274860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c6d79017-2b01-48f5-acd9-784ceffd89dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:17.275662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c6d79017-2b01-48f5-acd9-784ceffd89dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:17.276393 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c6d79017-2b01-48f5-acd9-784ceffd89dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:17.276849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c6d79017-2b01-48f5-acd9-784ceffd89dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:17.299930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:18:17.317219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c6d79017-2b01-48f5-acd9-784ceffd89dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058637 Aug 07 01:18:17.317908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 430/832] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:17 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:17.453538 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f8fa5b23-26aa-4b96-b129-199db76a32c6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:17.454601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f8fa5b23-26aa-4b96-b129-199db76a32c6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:17.461816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8fa5b23-26aa-4b96-b129-199db76a32c6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:17.462514 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8fa5b23-26aa-4b96-b129-199db76a32c6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:17.462961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8fa5b23-26aa-4b96-b129-199db76a32c6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:17.463686 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8fa5b23-26aa-4b96-b129-199db76a32c6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:17.464146 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8fa5b23-26aa-4b96-b129-199db76a32c6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:17.464599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8fa5b23-26aa-4b96-b129-199db76a32c6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:17.535189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f8fa5b23-26aa-4b96-b129-199db76a32c6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:17.543921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f8fa5b23-26aa-4b96-b129-199db76a32c6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.095741 Aug 07 01:18:17.544551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 403/833] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:17 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:18.331657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-344772ae-ae8e-45bf-971c-de0a14ca63ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:18.333997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-344772ae-ae8e-45bf-971c-de0a14ca63ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:18.341987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-344772ae-ae8e-45bf-971c-de0a14ca63ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:18.342488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-344772ae-ae8e-45bf-971c-de0a14ca63ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:18.342938 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-344772ae-ae8e-45bf-971c-de0a14ca63ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:18.343701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-344772ae-ae8e-45bf-971c-de0a14ca63ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:18.344130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-344772ae-ae8e-45bf-971c-de0a14ca63ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:18.344591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-344772ae-ae8e-45bf-971c-de0a14ca63ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:18.381693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-344772ae-ae8e-45bf-971c-de0a14ca63ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053574 Aug 07 01:18:18.382314 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 431/834] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:18 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:18.566750 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5656f334-487e-43a7-bc83-a33e390def84 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:18.567399 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5656f334-487e-43a7-bc83-a33e390def84 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:18.575450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5656f334-487e-43a7-bc83-a33e390def84 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:18.576075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5656f334-487e-43a7-bc83-a33e390def84 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:18.576612 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5656f334-487e-43a7-bc83-a33e390def84 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:18.577681 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5656f334-487e-43a7-bc83-a33e390def84 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:18.578812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5656f334-487e-43a7-bc83-a33e390def84 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:18.578812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5656f334-487e-43a7-bc83-a33e390def84 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:18.650509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5656f334-487e-43a7-bc83-a33e390def84 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:18.665099 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5656f334-487e-43a7-bc83-a33e390def84 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.102091 Aug 07 01:18:18.666166 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 404/835] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:18 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:19.398954 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-48a4dce8-52ce-4a1c-8ec6-511e13f3304d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:19.402488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-48a4dce8-52ce-4a1c-8ec6-511e13f3304d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:19.414738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-48a4dce8-52ce-4a1c-8ec6-511e13f3304d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:19.415615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-48a4dce8-52ce-4a1c-8ec6-511e13f3304d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:19.416473 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-48a4dce8-52ce-4a1c-8ec6-511e13f3304d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:19.417813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-48a4dce8-52ce-4a1c-8ec6-511e13f3304d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:19.418651 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-48a4dce8-52ce-4a1c-8ec6-511e13f3304d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:19.420073 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-48a4dce8-52ce-4a1c-8ec6-511e13f3304d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:19.478274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-48a4dce8-52ce-4a1c-8ec6-511e13f3304d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.085748 Aug 07 01:18:19.479432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 432/836] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:19 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:19.688340 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e3028bae-a3ef-49fc-b52b-b724d6a12a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:19.690245 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e3028bae-a3ef-49fc-b52b-b724d6a12a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:19.696098 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3028bae-a3ef-49fc-b52b-b724d6a12a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:19.696654 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3028bae-a3ef-49fc-b52b-b724d6a12a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:19.697100 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3028bae-a3ef-49fc-b52b-b724d6a12a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:19.697933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3028bae-a3ef-49fc-b52b-b724d6a12a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:19.698399 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3028bae-a3ef-49fc-b52b-b724d6a12a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:19.698830 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3028bae-a3ef-49fc-b52b-b724d6a12a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:19.773189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e3028bae-a3ef-49fc-b52b-b724d6a12a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:19.777193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:18:19.783489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e3028bae-a3ef-49fc-b52b-b724d6a12a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100111 Aug 07 01:18:19.784145 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 405/837] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:19 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:20.494999 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-923bc41a-cf50-450b-908e-9897d56b9bb3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:20.498839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-923bc41a-cf50-450b-908e-9897d56b9bb3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:20.506492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-923bc41a-cf50-450b-908e-9897d56b9bb3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:20.507187 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-923bc41a-cf50-450b-908e-9897d56b9bb3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:20.507750 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-923bc41a-cf50-450b-908e-9897d56b9bb3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:20.508616 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-923bc41a-cf50-450b-908e-9897d56b9bb3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:20.509124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-923bc41a-cf50-450b-908e-9897d56b9bb3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:20.509693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-923bc41a-cf50-450b-908e-9897d56b9bb3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:20.545854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-923bc41a-cf50-450b-908e-9897d56b9bb3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057472 Aug 07 01:18:20.546336 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 433/838] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:20 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:20.806602 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ecafbe16-8ed4-4b2b-8ad7-2106d9995eaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:20.807452 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ecafbe16-8ed4-4b2b-8ad7-2106d9995eaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:20.814856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecafbe16-8ed4-4b2b-8ad7-2106d9995eaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:20.815346 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecafbe16-8ed4-4b2b-8ad7-2106d9995eaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:20.815965 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecafbe16-8ed4-4b2b-8ad7-2106d9995eaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:20.816816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecafbe16-8ed4-4b2b-8ad7-2106d9995eaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:20.817267 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecafbe16-8ed4-4b2b-8ad7-2106d9995eaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:20.817785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecafbe16-8ed4-4b2b-8ad7-2106d9995eaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:20.892704 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ecafbe16-8ed4-4b2b-8ad7-2106d9995eaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:20.901575 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ecafbe16-8ed4-4b2b-8ad7-2106d9995eaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.099751 Aug 07 01:18:20.903179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 406/839] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:20 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:21.559035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0c2c112f-9566-4297-8c79-0d57844a23dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:21.561824 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0c2c112f-9566-4297-8c79-0d57844a23dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:21.569130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c2c112f-9566-4297-8c79-0d57844a23dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:21.569637 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c2c112f-9566-4297-8c79-0d57844a23dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:21.570090 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c2c112f-9566-4297-8c79-0d57844a23dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:21.570826 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c2c112f-9566-4297-8c79-0d57844a23dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:21.571292 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c2c112f-9566-4297-8c79-0d57844a23dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:21.571728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c2c112f-9566-4297-8c79-0d57844a23dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:21.607217 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0c2c112f-9566-4297-8c79-0d57844a23dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052323 Aug 07 01:18:21.607817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 434/840] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:21 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:21.936119 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7b3e9489-6200-44fe-9242-d6e91c79d1bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:21.936789 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7b3e9489-6200-44fe-9242-d6e91c79d1bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:21.944178 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7b3e9489-6200-44fe-9242-d6e91c79d1bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:21.944726 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7b3e9489-6200-44fe-9242-d6e91c79d1bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:21.945162 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7b3e9489-6200-44fe-9242-d6e91c79d1bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:21.945887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7b3e9489-6200-44fe-9242-d6e91c79d1bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:21.946349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7b3e9489-6200-44fe-9242-d6e91c79d1bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:21.946783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7b3e9489-6200-44fe-9242-d6e91c79d1bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:22.033556 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7b3e9489-6200-44fe-9242-d6e91c79d1bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:22.042415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7b3e9489-6200-44fe-9242-d6e91c79d1bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.111301 Aug 07 01:18:22.043060 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 407/841] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:21 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:22.619851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e524e66b-d8a2-49a0-8b74-675a9f048743 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:22.622243 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e524e66b-d8a2-49a0-8b74-675a9f048743 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:22.629918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e524e66b-d8a2-49a0-8b74-675a9f048743 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:22.630503 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e524e66b-d8a2-49a0-8b74-675a9f048743 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:22.631182 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e524e66b-d8a2-49a0-8b74-675a9f048743 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:22.631969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e524e66b-d8a2-49a0-8b74-675a9f048743 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:22.632545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e524e66b-d8a2-49a0-8b74-675a9f048743 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:22.633028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e524e66b-d8a2-49a0-8b74-675a9f048743 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:22.669994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e524e66b-d8a2-49a0-8b74-675a9f048743 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054361 Aug 07 01:18:22.670633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 435/842] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:22 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:23.064215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-48d0c8ed-ca56-4242-9ad7-2bb94ba66b1b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:23.065067 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-48d0c8ed-ca56-4242-9ad7-2bb94ba66b1b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:23.071783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-48d0c8ed-ca56-4242-9ad7-2bb94ba66b1b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:23.072435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-48d0c8ed-ca56-4242-9ad7-2bb94ba66b1b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:23.072905 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-48d0c8ed-ca56-4242-9ad7-2bb94ba66b1b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:23.073663 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-48d0c8ed-ca56-4242-9ad7-2bb94ba66b1b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:23.074122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-48d0c8ed-ca56-4242-9ad7-2bb94ba66b1b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:23.074744 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-48d0c8ed-ca56-4242-9ad7-2bb94ba66b1b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:23.328820 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-48d0c8ed-ca56-4242-9ad7-2bb94ba66b1b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:23.337604 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-48d0c8ed-ca56-4242-9ad7-2bb94ba66b1b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.277192 Aug 07 01:18:23.338357 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 408/843] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:23 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:23.683865 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ffd5c803-6e07-4e77-8190-d60a72c4d6b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:23.686288 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ffd5c803-6e07-4e77-8190-d60a72c4d6b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:23.694088 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ffd5c803-6e07-4e77-8190-d60a72c4d6b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:23.694725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ffd5c803-6e07-4e77-8190-d60a72c4d6b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:23.695250 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ffd5c803-6e07-4e77-8190-d60a72c4d6b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:23.696217 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ffd5c803-6e07-4e77-8190-d60a72c4d6b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:23.696873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ffd5c803-6e07-4e77-8190-d60a72c4d6b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:23.697463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ffd5c803-6e07-4e77-8190-d60a72c4d6b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:23.734861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ffd5c803-6e07-4e77-8190-d60a72c4d6b6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055141 Aug 07 01:18:23.735464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 436/844] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:23 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:24.364119 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-70d86c0f-4411-4770-ad85-5f2d9b5641d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:24.364984 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-70d86c0f-4411-4770-ad85-5f2d9b5641d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:24.374606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-70d86c0f-4411-4770-ad85-5f2d9b5641d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:24.375110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-70d86c0f-4411-4770-ad85-5f2d9b5641d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:24.375556 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-70d86c0f-4411-4770-ad85-5f2d9b5641d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:24.376546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-70d86c0f-4411-4770-ad85-5f2d9b5641d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:24.377358 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-70d86c0f-4411-4770-ad85-5f2d9b5641d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:24.378017 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-70d86c0f-4411-4770-ad85-5f2d9b5641d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:24.477126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-70d86c0f-4411-4770-ad85-5f2d9b5641d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:24.490448 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-70d86c0f-4411-4770-ad85-5f2d9b5641d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.132262 Aug 07 01:18:24.491539 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 409/845] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:24 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:24.751495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f878b987-9eba-423c-98fb-c460fb11598e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:24.757736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f878b987-9eba-423c-98fb-c460fb11598e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:24.769535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f878b987-9eba-423c-98fb-c460fb11598e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:24.770183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f878b987-9eba-423c-98fb-c460fb11598e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:24.770855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f878b987-9eba-423c-98fb-c460fb11598e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:24.772140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f878b987-9eba-423c-98fb-c460fb11598e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:24.773028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f878b987-9eba-423c-98fb-c460fb11598e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:24.773672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f878b987-9eba-423c-98fb-c460fb11598e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:24.825652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f878b987-9eba-423c-98fb-c460fb11598e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080423 Aug 07 01:18:24.827008 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 437/846] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:24 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:25.532781 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-661970e0-1949-44f0-aaf2-b0465a986312 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:25.533805 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-661970e0-1949-44f0-aaf2-b0465a986312 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:25.563558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-661970e0-1949-44f0-aaf2-b0465a986312 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:25.564170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-661970e0-1949-44f0-aaf2-b0465a986312 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:25.564707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-661970e0-1949-44f0-aaf2-b0465a986312 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:25.565510 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-661970e0-1949-44f0-aaf2-b0465a986312 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:25.566048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-661970e0-1949-44f0-aaf2-b0465a986312 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:25.566518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-661970e0-1949-44f0-aaf2-b0465a986312 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:25.677285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-661970e0-1949-44f0-aaf2-b0465a986312 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:25.691159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-661970e0-1949-44f0-aaf2-b0465a986312 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.165352 Aug 07 01:18:25.691962 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 410/847] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:25 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:25.847915 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9d139d49-24ed-44e3-b445-4755bd04fddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:25.851608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9d139d49-24ed-44e3-b445-4755bd04fddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:25.865441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d139d49-24ed-44e3-b445-4755bd04fddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:25.866331 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d139d49-24ed-44e3-b445-4755bd04fddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:25.867117 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d139d49-24ed-44e3-b445-4755bd04fddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:25.868464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d139d49-24ed-44e3-b445-4755bd04fddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:25.869318 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d139d49-24ed-44e3-b445-4755bd04fddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:25.870056 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d139d49-24ed-44e3-b445-4755bd04fddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:25.931629 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9d139d49-24ed-44e3-b445-4755bd04fddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.091668 Aug 07 01:18:25.932756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 438/848] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:25 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:26.735653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8bede3f3-4572-499b-a0c1-3f5c4092dc28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:26.736777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8bede3f3-4572-499b-a0c1-3f5c4092dc28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:26.749620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8bede3f3-4572-499b-a0c1-3f5c4092dc28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:26.750453 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8bede3f3-4572-499b-a0c1-3f5c4092dc28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:26.751197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8bede3f3-4572-499b-a0c1-3f5c4092dc28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:26.752531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8bede3f3-4572-499b-a0c1-3f5c4092dc28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:26.753379 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8bede3f3-4572-499b-a0c1-3f5c4092dc28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:26.754137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8bede3f3-4572-499b-a0c1-3f5c4092dc28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:26.872683 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8bede3f3-4572-499b-a0c1-3f5c4092dc28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:26.886181 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8bede3f3-4572-499b-a0c1-3f5c4092dc28 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.157323 Aug 07 01:18:26.887095 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 411/849] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:26 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:26.952994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6ba5d3c3-33f1-4d14-9637-c53e9fe7d7e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:26.956884 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6ba5d3c3-33f1-4d14-9637-c53e9fe7d7e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:26.969625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba5d3c3-33f1-4d14-9637-c53e9fe7d7e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:26.970962 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba5d3c3-33f1-4d14-9637-c53e9fe7d7e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:26.971269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba5d3c3-33f1-4d14-9637-c53e9fe7d7e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:26.972647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba5d3c3-33f1-4d14-9637-c53e9fe7d7e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:26.973500 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba5d3c3-33f1-4d14-9637-c53e9fe7d7e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:26.974269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba5d3c3-33f1-4d14-9637-c53e9fe7d7e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:27.029384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6ba5d3c3-33f1-4d14-9637-c53e9fe7d7e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.083576 Aug 07 01:18:27.030426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 439/850] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:26 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:27.923256 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-133e855a-f2e4-43bc-8cfd-0a87e8a77ef6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:27.924204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-133e855a-f2e4-43bc-8cfd-0a87e8a77ef6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:27.939421 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-133e855a-f2e4-43bc-8cfd-0a87e8a77ef6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:27.939918 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-133e855a-f2e4-43bc-8cfd-0a87e8a77ef6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:27.940391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-133e855a-f2e4-43bc-8cfd-0a87e8a77ef6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:27.941133 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-133e855a-f2e4-43bc-8cfd-0a87e8a77ef6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:27.941598 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-133e855a-f2e4-43bc-8cfd-0a87e8a77ef6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:27.942031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-133e855a-f2e4-43bc-8cfd-0a87e8a77ef6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:28.026248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-133e855a-f2e4-43bc-8cfd-0a87e8a77ef6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:28.036591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-133e855a-f2e4-43bc-8cfd-0a87e8a77ef6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.121122 Aug 07 01:18:28.037427 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 412/851] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:27 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:28.050505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-05f78530-6295-456e-80a3-725f2610db4e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:28.054551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-05f78530-6295-456e-80a3-725f2610db4e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:28.064698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05f78530-6295-456e-80a3-725f2610db4e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:28.065200 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05f78530-6295-456e-80a3-725f2610db4e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:28.065655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05f78530-6295-456e-80a3-725f2610db4e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:28.066381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05f78530-6295-456e-80a3-725f2610db4e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:28.066844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05f78530-6295-456e-80a3-725f2610db4e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:28.067272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05f78530-6295-456e-80a3-725f2610db4e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:28.127389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-05f78530-6295-456e-80a3-725f2610db4e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.084761 Aug 07 01:18:28.132613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 440/852] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:28 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:18:29.074952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ff0d0059-9d2b-4ad2-b0aa-660b8ed0ea7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:29.076095 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ff0d0059-9d2b-4ad2-b0aa-660b8ed0ea7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:29.088859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff0d0059-9d2b-4ad2-b0aa-660b8ed0ea7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:29.089738 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff0d0059-9d2b-4ad2-b0aa-660b8ed0ea7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:29.091463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff0d0059-9d2b-4ad2-b0aa-660b8ed0ea7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:29.093066 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff0d0059-9d2b-4ad2-b0aa-660b8ed0ea7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:29.093912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff0d0059-9d2b-4ad2-b0aa-660b8ed0ea7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:29.094862 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff0d0059-9d2b-4ad2-b0aa-660b8ed0ea7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:29.143864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-743da03c-a469-4150-89bf-a8ce1a692526 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:29.147222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-743da03c-a469-4150-89bf-a8ce1a692526 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:29.157795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-743da03c-a469-4150-89bf-a8ce1a692526 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:29.158517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-743da03c-a469-4150-89bf-a8ce1a692526 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:29.159152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-743da03c-a469-4150-89bf-a8ce1a692526 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:29.160211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-743da03c-a469-4150-89bf-a8ce1a692526 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:29.160940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-743da03c-a469-4150-89bf-a8ce1a692526 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:29.161544 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-743da03c-a469-4150-89bf-a8ce1a692526 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:29.213156 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-743da03c-a469-4150-89bf-a8ce1a692526 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.075066 Aug 07 01:18:29.214055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 441/853] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:29 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:18:29.216046 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ff0d0059-9d2b-4ad2-b0aa-660b8ed0ea7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:29.229378 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ff0d0059-9d2b-4ad2-b0aa-660b8ed0ea7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.162187 Aug 07 01:18:29.230262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 413/854] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:29 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:30.234458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d452c8eb-9965-41af-b131-f2c092174dce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:30.238309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d452c8eb-9965-41af-b131-f2c092174dce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:30.244330 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=56 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:18:30.251753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d452c8eb-9965-41af-b131-f2c092174dce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:30.252620 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d452c8eb-9965-41af-b131-f2c092174dce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:30.253408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d452c8eb-9965-41af-b131-f2c092174dce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:30.254830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d452c8eb-9965-41af-b131-f2c092174dce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:30.255699 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d452c8eb-9965-41af-b131-f2c092174dce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:30.256498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d452c8eb-9965-41af-b131-f2c092174dce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:30.261461 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-10608452-7d2f-4d38-bdc3-744d6bea4663 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:30.262920 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-10608452-7d2f-4d38-bdc3-744d6bea4663 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:30.271302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10608452-7d2f-4d38-bdc3-744d6bea4663 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:30.271943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10608452-7d2f-4d38-bdc3-744d6bea4663 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:30.272555 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10608452-7d2f-4d38-bdc3-744d6bea4663 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:30.273573 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10608452-7d2f-4d38-bdc3-744d6bea4663 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:30.274144 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10608452-7d2f-4d38-bdc3-744d6bea4663 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:30.274790 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10608452-7d2f-4d38-bdc3-744d6bea4663 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:30.315298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d452c8eb-9965-41af-b131-f2c092174dce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.087865 Aug 07 01:18:30.316205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 442/855] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:30 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:18:30.359670 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-10608452-7d2f-4d38-bdc3-744d6bea4663 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:30.371225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-10608452-7d2f-4d38-bdc3-744d6bea4663 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.116056 Aug 07 01:18:30.372650 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 414/856] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:30 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:31.332601 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-422308f5-438e-4f4f-90dc-7745f2f4030d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:31.335044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-422308f5-438e-4f4f-90dc-7745f2f4030d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:31.342757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-422308f5-438e-4f4f-90dc-7745f2f4030d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:31.343406 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-422308f5-438e-4f4f-90dc-7745f2f4030d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:31.343979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-422308f5-438e-4f4f-90dc-7745f2f4030d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:31.344862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-422308f5-438e-4f4f-90dc-7745f2f4030d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:31.345510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-422308f5-438e-4f4f-90dc-7745f2f4030d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:31.346042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-422308f5-438e-4f4f-90dc-7745f2f4030d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:31.381931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-422308f5-438e-4f4f-90dc-7745f2f4030d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053485 Aug 07 01:18:31.382609 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 443/857] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:31 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:31.395109 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-dcfaf909-6526-48c9-b280-4261ac80bf63 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:31.395894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-dcfaf909-6526-48c9-b280-4261ac80bf63 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:31.398641 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=54 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:18:31.405940 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dcfaf909-6526-48c9-b280-4261ac80bf63 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:31.406448 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dcfaf909-6526-48c9-b280-4261ac80bf63 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:31.406975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dcfaf909-6526-48c9-b280-4261ac80bf63 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:31.407709 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dcfaf909-6526-48c9-b280-4261ac80bf63 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:31.408167 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dcfaf909-6526-48c9-b280-4261ac80bf63 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:31.408746 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dcfaf909-6526-48c9-b280-4261ac80bf63 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:31.480401 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dcfaf909-6526-48c9-b280-4261ac80bf63 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:31.489157 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-dcfaf909-6526-48c9-b280-4261ac80bf63 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.098161 Aug 07 01:18:31.489766 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 415/858] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:31 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:32.395004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7c15acaf-4472-4192-b380-4ed1cc44bae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:32.397380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7c15acaf-4472-4192-b380-4ed1cc44bae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:32.404250 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c15acaf-4472-4192-b380-4ed1cc44bae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:32.404751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c15acaf-4472-4192-b380-4ed1cc44bae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:32.405192 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c15acaf-4472-4192-b380-4ed1cc44bae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:32.405912 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c15acaf-4472-4192-b380-4ed1cc44bae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:32.406370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c15acaf-4472-4192-b380-4ed1cc44bae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:32.406800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c15acaf-4472-4192-b380-4ed1cc44bae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:32.444794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7c15acaf-4472-4192-b380-4ed1cc44bae5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053612 Aug 07 01:18:32.445287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 444/859] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:32 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:32.510494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-14928594-6eaa-44e9-abae-aa55acecf192 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:32.511134 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-14928594-6eaa-44e9-abae-aa55acecf192 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:32.518614 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-14928594-6eaa-44e9-abae-aa55acecf192 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:32.519108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-14928594-6eaa-44e9-abae-aa55acecf192 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:32.519558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-14928594-6eaa-44e9-abae-aa55acecf192 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:32.520320 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-14928594-6eaa-44e9-abae-aa55acecf192 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:32.520779 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-14928594-6eaa-44e9-abae-aa55acecf192 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:32.521210 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-14928594-6eaa-44e9-abae-aa55acecf192 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:32.589599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-14928594-6eaa-44e9-abae-aa55acecf192 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:32.598797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-14928594-6eaa-44e9-abae-aa55acecf192 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.092174 Aug 07 01:18:32.599451 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 416/860] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:32 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:33.457777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dafb7b7b-d666-4139-9abc-eea16ee1f11a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:33.460091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dafb7b7b-d666-4139-9abc-eea16ee1f11a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:33.467371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dafb7b7b-d666-4139-9abc-eea16ee1f11a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:33.467861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dafb7b7b-d666-4139-9abc-eea16ee1f11a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:33.468352 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dafb7b7b-d666-4139-9abc-eea16ee1f11a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:33.469069 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dafb7b7b-d666-4139-9abc-eea16ee1f11a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:33.469534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dafb7b7b-d666-4139-9abc-eea16ee1f11a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:33.469970 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dafb7b7b-d666-4139-9abc-eea16ee1f11a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:33.507127 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dafb7b7b-d666-4139-9abc-eea16ee1f11a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053743 Aug 07 01:18:33.507726 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 445/861] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:33 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:18:33.626118 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a782dc34-2f0e-4452-b2d5-00d0d591fbe8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:33.626760 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a782dc34-2f0e-4452-b2d5-00d0d591fbe8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:33.633773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a782dc34-2f0e-4452-b2d5-00d0d591fbe8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:33.634267 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a782dc34-2f0e-4452-b2d5-00d0d591fbe8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:33.634711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a782dc34-2f0e-4452-b2d5-00d0d591fbe8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:33.635432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a782dc34-2f0e-4452-b2d5-00d0d591fbe8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:33.635889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a782dc34-2f0e-4452-b2d5-00d0d591fbe8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:33.636356 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a782dc34-2f0e-4452-b2d5-00d0d591fbe8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:33.706263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a782dc34-2f0e-4452-b2d5-00d0d591fbe8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:33.715620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a782dc34-2f0e-4452-b2d5-00d0d591fbe8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.093758 Aug 07 01:18:33.716116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 417/862] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:33 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:34.520168 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-55bf269c-778b-4a76-891a-472a6f00fb92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:34.524737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-55bf269c-778b-4a76-891a-472a6f00fb92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:34.537404 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-55bf269c-778b-4a76-891a-472a6f00fb92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:34.538184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-55bf269c-778b-4a76-891a-472a6f00fb92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:34.539029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-55bf269c-778b-4a76-891a-472a6f00fb92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:34.540215 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-55bf269c-778b-4a76-891a-472a6f00fb92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:34.541045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-55bf269c-778b-4a76-891a-472a6f00fb92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:34.541764 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-55bf269c-778b-4a76-891a-472a6f00fb92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:34.599584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-55bf269c-778b-4a76-891a-472a6f00fb92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.083402 Aug 07 01:18:34.600673 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 446/863] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:34 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:34.737994 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-85659197-0d0b-454a-8058-2800c7211a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:34.738759 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-85659197-0d0b-454a-8058-2800c7211a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:34.745803 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-85659197-0d0b-454a-8058-2800c7211a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:34.747257 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-85659197-0d0b-454a-8058-2800c7211a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:34.747707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-85659197-0d0b-454a-8058-2800c7211a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:34.748493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-85659197-0d0b-454a-8058-2800c7211a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:34.748959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-85659197-0d0b-454a-8058-2800c7211a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:34.749393 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-85659197-0d0b-454a-8058-2800c7211a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:34.834558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-85659197-0d0b-454a-8058-2800c7211a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:34.843123 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-85659197-0d0b-454a-8058-2800c7211a6e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.109437 Aug 07 01:18:34.843727 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 418/864] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:34 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:35.614545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-26c74c53-2e77-4612-956a-8bc880ad80f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:35.616864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-26c74c53-2e77-4612-956a-8bc880ad80f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:35.626166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26c74c53-2e77-4612-956a-8bc880ad80f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:35.626779 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26c74c53-2e77-4612-956a-8bc880ad80f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:35.627383 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26c74c53-2e77-4612-956a-8bc880ad80f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:35.628462 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26c74c53-2e77-4612-956a-8bc880ad80f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:35.629066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26c74c53-2e77-4612-956a-8bc880ad80f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:35.629559 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26c74c53-2e77-4612-956a-8bc880ad80f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:35.634350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:18:35.670329 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-26c74c53-2e77-4612-956a-8bc880ad80f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060004 Aug 07 01:18:35.671194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 447/865] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:35 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:35.865603 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1c63da3f-faab-4182-8fbd-32795fdf77bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:35.866204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1c63da3f-faab-4182-8fbd-32795fdf77bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:35.873348 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1c63da3f-faab-4182-8fbd-32795fdf77bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:35.873843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1c63da3f-faab-4182-8fbd-32795fdf77bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:35.874292 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1c63da3f-faab-4182-8fbd-32795fdf77bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:35.875034 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1c63da3f-faab-4182-8fbd-32795fdf77bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:35.875493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1c63da3f-faab-4182-8fbd-32795fdf77bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:35.875926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1c63da3f-faab-4182-8fbd-32795fdf77bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:35.959227 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1c63da3f-faab-4182-8fbd-32795fdf77bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:35.968223 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1c63da3f-faab-4182-8fbd-32795fdf77bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.107250 Aug 07 01:18:35.968955 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 419/866] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:35 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:36.686854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6281bd36-7f15-48b7-a48d-a1506ea889d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:36.689246 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6281bd36-7f15-48b7-a48d-a1506ea889d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:36.696628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6281bd36-7f15-48b7-a48d-a1506ea889d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:36.697221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6281bd36-7f15-48b7-a48d-a1506ea889d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:36.697698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6281bd36-7f15-48b7-a48d-a1506ea889d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:36.698466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6281bd36-7f15-48b7-a48d-a1506ea889d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:36.698963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6281bd36-7f15-48b7-a48d-a1506ea889d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:36.700114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6281bd36-7f15-48b7-a48d-a1506ea889d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:36.749166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6281bd36-7f15-48b7-a48d-a1506ea889d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066587 Aug 07 01:18:36.750195 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 448/867] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:36 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:18:36.991405 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ef69d495-2509-4ce0-94b2-950d9912f868 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:36.992648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ef69d495-2509-4ce0-94b2-950d9912f868 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:37.004539 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef69d495-2509-4ce0-94b2-950d9912f868 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:37.005399 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef69d495-2509-4ce0-94b2-950d9912f868 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:37.006167 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef69d495-2509-4ce0-94b2-950d9912f868 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:37.007480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef69d495-2509-4ce0-94b2-950d9912f868 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:37.008343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef69d495-2509-4ce0-94b2-950d9912f868 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:37.009098 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef69d495-2509-4ce0-94b2-950d9912f868 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:37.084192 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ef69d495-2509-4ce0-94b2-950d9912f868 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:37.094548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ef69d495-2509-4ce0-94b2-950d9912f868 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.108115 Aug 07 01:18:37.095194 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 420/868] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:36 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:37.765888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7302ac0c-697c-438a-81ea-24e7b322be2d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:37.768203 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7302ac0c-697c-438a-81ea-24e7b322be2d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:37.775396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7302ac0c-697c-438a-81ea-24e7b322be2d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:37.775932 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7302ac0c-697c-438a-81ea-24e7b322be2d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:37.776490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7302ac0c-697c-438a-81ea-24e7b322be2d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:37.778011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7302ac0c-697c-438a-81ea-24e7b322be2d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:37.778707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7302ac0c-697c-438a-81ea-24e7b322be2d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:37.779261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7302ac0c-697c-438a-81ea-24e7b322be2d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:37.827285 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7302ac0c-697c-438a-81ea-24e7b322be2d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065469 Aug 07 01:18:37.828377 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 449/869] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:37 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:38.117375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-311d02e8-27ea-4026-8545-89b934465fee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:38.118212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-311d02e8-27ea-4026-8545-89b934465fee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:38.126864 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-311d02e8-27ea-4026-8545-89b934465fee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:38.127642 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-311d02e8-27ea-4026-8545-89b934465fee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:38.128375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-311d02e8-27ea-4026-8545-89b934465fee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:38.129582 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-311d02e8-27ea-4026-8545-89b934465fee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:38.130345 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-311d02e8-27ea-4026-8545-89b934465fee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:38.130917 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-311d02e8-27ea-4026-8545-89b934465fee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:38.245356 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-311d02e8-27ea-4026-8545-89b934465fee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:38.254209 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-311d02e8-27ea-4026-8545-89b934465fee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.140744 Aug 07 01:18:38.255049 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 421/870] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:38 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:38.844816 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c822e079-1141-4787-a465-c1a483b8700b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:38.847417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c822e079-1141-4787-a465-c1a483b8700b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:38.855179 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c822e079-1141-4787-a465-c1a483b8700b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:38.855734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c822e079-1141-4787-a465-c1a483b8700b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:38.856786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c822e079-1141-4787-a465-c1a483b8700b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:38.857724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c822e079-1141-4787-a465-c1a483b8700b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:38.858207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c822e079-1141-4787-a465-c1a483b8700b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:38.858643 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c822e079-1141-4787-a465-c1a483b8700b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:38.895738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c822e079-1141-4787-a465-c1a483b8700b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056170 Aug 07 01:18:38.896482 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 450/871] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:38 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:39.295468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9af1b7a9-82ac-4ea2-b551-8ec3ecffb93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:39.296555 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9af1b7a9-82ac-4ea2-b551-8ec3ecffb93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:39.308305 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9af1b7a9-82ac-4ea2-b551-8ec3ecffb93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:39.309772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9af1b7a9-82ac-4ea2-b551-8ec3ecffb93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:39.310532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9af1b7a9-82ac-4ea2-b551-8ec3ecffb93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:39.311759 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9af1b7a9-82ac-4ea2-b551-8ec3ecffb93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:39.312670 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9af1b7a9-82ac-4ea2-b551-8ec3ecffb93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:39.313413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9af1b7a9-82ac-4ea2-b551-8ec3ecffb93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:39.338708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:18:39.389969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9af1b7a9-82ac-4ea2-b551-8ec3ecffb93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:39.398359 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9af1b7a9-82ac-4ea2-b551-8ec3ecffb93c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.110475 Aug 07 01:18:39.399143 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 422/872] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:39 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:39.909650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4c35970a-6097-4d08-ba9c-53db1cc6a1d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:39.911943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4c35970a-6097-4d08-ba9c-53db1cc6a1d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:39.918478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4c35970a-6097-4d08-ba9c-53db1cc6a1d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:39.918971 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4c35970a-6097-4d08-ba9c-53db1cc6a1d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:39.919413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4c35970a-6097-4d08-ba9c-53db1cc6a1d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:39.920128 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4c35970a-6097-4d08-ba9c-53db1cc6a1d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:39.920709 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4c35970a-6097-4d08-ba9c-53db1cc6a1d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:39.921139 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4c35970a-6097-4d08-ba9c-53db1cc6a1d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:39.956084 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4c35970a-6097-4d08-ba9c-53db1cc6a1d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050563 Aug 07 01:18:39.956768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 451/873] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:39 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:40.421449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-eeb8e1e4-2a5f-4d7c-aa0c-52047a4c96a7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:40.422083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-eeb8e1e4-2a5f-4d7c-aa0c-52047a4c96a7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:40.432133 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eeb8e1e4-2a5f-4d7c-aa0c-52047a4c96a7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:40.433093 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eeb8e1e4-2a5f-4d7c-aa0c-52047a4c96a7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:40.434403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eeb8e1e4-2a5f-4d7c-aa0c-52047a4c96a7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:40.435673 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eeb8e1e4-2a5f-4d7c-aa0c-52047a4c96a7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:40.436524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eeb8e1e4-2a5f-4d7c-aa0c-52047a4c96a7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:40.437285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eeb8e1e4-2a5f-4d7c-aa0c-52047a4c96a7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:40.528745 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-eeb8e1e4-2a5f-4d7c-aa0c-52047a4c96a7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:40.538717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-eeb8e1e4-2a5f-4d7c-aa0c-52047a4c96a7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.121085 Aug 07 01:18:40.539382 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 423/874] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:40 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:40.970479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c0a7fa13-8819-4fd2-8bfb-5211c9be7525 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:40.972657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c0a7fa13-8819-4fd2-8bfb-5211c9be7525 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:40.980533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c0a7fa13-8819-4fd2-8bfb-5211c9be7525 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:40.981035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c0a7fa13-8819-4fd2-8bfb-5211c9be7525 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:40.981483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c0a7fa13-8819-4fd2-8bfb-5211c9be7525 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:40.982236 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c0a7fa13-8819-4fd2-8bfb-5211c9be7525 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:40.982703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c0a7fa13-8819-4fd2-8bfb-5211c9be7525 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:40.983142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c0a7fa13-8819-4fd2-8bfb-5211c9be7525 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:41.033533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c0a7fa13-8819-4fd2-8bfb-5211c9be7525 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067609 Aug 07 01:18:41.034612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 452/875] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:40 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:18:41.561344 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c26ecb28-57b6-4ba3-bc25-fc2873487a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:41.561925 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c26ecb28-57b6-4ba3-bc25-fc2873487a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:41.569769 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c26ecb28-57b6-4ba3-bc25-fc2873487a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:41.570346 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c26ecb28-57b6-4ba3-bc25-fc2873487a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:41.570810 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c26ecb28-57b6-4ba3-bc25-fc2873487a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:41.571834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c26ecb28-57b6-4ba3-bc25-fc2873487a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:41.572498 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c26ecb28-57b6-4ba3-bc25-fc2873487a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:41.572924 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c26ecb28-57b6-4ba3-bc25-fc2873487a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:41.654968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c26ecb28-57b6-4ba3-bc25-fc2873487a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:41.669984 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c26ecb28-57b6-4ba3-bc25-fc2873487a86 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.113055 Aug 07 01:18:41.671354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 424/876] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:41 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:42.051563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-96f1415b-f4ac-4d77-9550-49d7025ee732 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:42.055223 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-96f1415b-f4ac-4d77-9550-49d7025ee732 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:42.064062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-96f1415b-f4ac-4d77-9550-49d7025ee732 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:42.064948 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-96f1415b-f4ac-4d77-9550-49d7025ee732 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:42.065412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-96f1415b-f4ac-4d77-9550-49d7025ee732 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:42.066169 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-96f1415b-f4ac-4d77-9550-49d7025ee732 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:42.066768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-96f1415b-f4ac-4d77-9550-49d7025ee732 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:42.067277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-96f1415b-f4ac-4d77-9550-49d7025ee732 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:42.109147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-96f1415b-f4ac-4d77-9550-49d7025ee732 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063153 Aug 07 01:18:42.109759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 453/877] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:42 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:42.709018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-37f01cbc-2d7f-4af1-b686-7deb9e25c857 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:42.709874 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-37f01cbc-2d7f-4af1-b686-7deb9e25c857 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:42.718363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37f01cbc-2d7f-4af1-b686-7deb9e25c857 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:42.718887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37f01cbc-2d7f-4af1-b686-7deb9e25c857 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:42.719321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37f01cbc-2d7f-4af1-b686-7deb9e25c857 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:42.720063 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37f01cbc-2d7f-4af1-b686-7deb9e25c857 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:42.720673 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37f01cbc-2d7f-4af1-b686-7deb9e25c857 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:42.721211 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37f01cbc-2d7f-4af1-b686-7deb9e25c857 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:42.838163 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-37f01cbc-2d7f-4af1-b686-7deb9e25c857 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:42.848994 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-37f01cbc-2d7f-4af1-b686-7deb9e25c857 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.148918 Aug 07 01:18:42.850097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 425/878] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:42 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:43.130882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1541a028-f97c-408c-946d-525a117bd66e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:43.135596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1541a028-f97c-408c-946d-525a117bd66e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:43.151613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1541a028-f97c-408c-946d-525a117bd66e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:43.152764 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1541a028-f97c-408c-946d-525a117bd66e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:43.153435 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1541a028-f97c-408c-946d-525a117bd66e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:43.157724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1541a028-f97c-408c-946d-525a117bd66e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:43.157724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1541a028-f97c-408c-946d-525a117bd66e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:43.157724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1541a028-f97c-408c-946d-525a117bd66e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:43.205083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1541a028-f97c-408c-946d-525a117bd66e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.082026 Aug 07 01:18:43.205800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 454/879] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:43 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:44.063112 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-67ec13cb-ecdd-4c17-a456-89e926200249 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:44.063881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-67ec13cb-ecdd-4c17-a456-89e926200249 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:44.071981 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67ec13cb-ecdd-4c17-a456-89e926200249 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:44.072848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67ec13cb-ecdd-4c17-a456-89e926200249 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:44.073794 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67ec13cb-ecdd-4c17-a456-89e926200249 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:44.075239 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67ec13cb-ecdd-4c17-a456-89e926200249 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:44.075841 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67ec13cb-ecdd-4c17-a456-89e926200249 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:44.076368 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-67ec13cb-ecdd-4c17-a456-89e926200249 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:44.164916 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-67ec13cb-ecdd-4c17-a456-89e926200249 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:44.174529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-67ec13cb-ecdd-4c17-a456-89e926200249 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.294270 Aug 07 01:18:44.175238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 426/880] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:43 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:44.223341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f1d0c87d-c620-4a7c-8370-730ccb6701d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:44.226679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f1d0c87d-c620-4a7c-8370-730ccb6701d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:44.236171 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f1d0c87d-c620-4a7c-8370-730ccb6701d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:44.236779 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f1d0c87d-c620-4a7c-8370-730ccb6701d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:44.237399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f1d0c87d-c620-4a7c-8370-730ccb6701d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:44.238200 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f1d0c87d-c620-4a7c-8370-730ccb6701d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:44.238776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f1d0c87d-c620-4a7c-8370-730ccb6701d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:44.239314 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f1d0c87d-c620-4a7c-8370-730ccb6701d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:44.301120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f1d0c87d-c620-4a7c-8370-730ccb6701d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.083410 Aug 07 01:18:44.306478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 455/881] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:44 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:45.195551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b171d142-5ef4-4526-936d-282234bb0ecf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:45.196200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b171d142-5ef4-4526-936d-282234bb0ecf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:45.204097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b171d142-5ef4-4526-936d-282234bb0ecf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:45.204623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b171d142-5ef4-4526-936d-282234bb0ecf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:45.205062 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b171d142-5ef4-4526-936d-282234bb0ecf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:45.205784 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b171d142-5ef4-4526-936d-282234bb0ecf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:45.206235 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b171d142-5ef4-4526-936d-282234bb0ecf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:45.206663 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b171d142-5ef4-4526-936d-282234bb0ecf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:45.295139 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b171d142-5ef4-4526-936d-282234bb0ecf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:45.304696 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b171d142-5ef4-4526-936d-282234bb0ecf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.113051 Aug 07 01:18:45.305347 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 427/882] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:45 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:45.318978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dec28b9e-1af7-4ee0-a1dc-35a9aab47264 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:45.322764 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dec28b9e-1af7-4ee0-a1dc-35a9aab47264 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:45.334837 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dec28b9e-1af7-4ee0-a1dc-35a9aab47264 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:45.335590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dec28b9e-1af7-4ee0-a1dc-35a9aab47264 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:45.336416 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dec28b9e-1af7-4ee0-a1dc-35a9aab47264 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:45.337789 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dec28b9e-1af7-4ee0-a1dc-35a9aab47264 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:45.338652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dec28b9e-1af7-4ee0-a1dc-35a9aab47264 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:45.340060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dec28b9e-1af7-4ee0-a1dc-35a9aab47264 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:45.396667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dec28b9e-1af7-4ee0-a1dc-35a9aab47264 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.084843 Aug 07 01:18:45.397368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 456/883] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:45 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:46.326600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-acb2c4c2-fc12-4cbd-a59a-6cce72123657 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:46.327240 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-acb2c4c2-fc12-4cbd-a59a-6cce72123657 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:46.338354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acb2c4c2-fc12-4cbd-a59a-6cce72123657 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:46.339232 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acb2c4c2-fc12-4cbd-a59a-6cce72123657 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:46.340214 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acb2c4c2-fc12-4cbd-a59a-6cce72123657 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:46.341101 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acb2c4c2-fc12-4cbd-a59a-6cce72123657 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:46.341569 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acb2c4c2-fc12-4cbd-a59a-6cce72123657 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:46.341999 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acb2c4c2-fc12-4cbd-a59a-6cce72123657 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:46.410334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9bb34bfc-44fe-4a89-822b-259faa117f3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:46.413039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9bb34bfc-44fe-4a89-822b-259faa117f3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:46.422507 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9bb34bfc-44fe-4a89-822b-259faa117f3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:46.423773 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9bb34bfc-44fe-4a89-822b-259faa117f3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:46.424067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9bb34bfc-44fe-4a89-822b-259faa117f3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:46.424988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9bb34bfc-44fe-4a89-822b-259faa117f3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:46.425597 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9bb34bfc-44fe-4a89-822b-259faa117f3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:46.426094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9bb34bfc-44fe-4a89-822b-259faa117f3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:46.427516 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-acb2c4c2-fc12-4cbd-a59a-6cce72123657 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:46.440711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-acb2c4c2-fc12-4cbd-a59a-6cce72123657 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.118165 Aug 07 01:18:46.441736 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 428/884] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:46 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:46.475865 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9bb34bfc-44fe-4a89-822b-259faa117f3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069624 Aug 07 01:18:46.476975 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 457/885] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:46 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:18:47.474364 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3b86a35f-c113-4c27-823e-137a0e244f5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:47.475353 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3b86a35f-c113-4c27-823e-137a0e244f5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:47.483340 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b86a35f-c113-4c27-823e-137a0e244f5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:47.484025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b86a35f-c113-4c27-823e-137a0e244f5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:47.484699 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b86a35f-c113-4c27-823e-137a0e244f5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:47.485639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b86a35f-c113-4c27-823e-137a0e244f5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:47.486323 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b86a35f-c113-4c27-823e-137a0e244f5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:47.486818 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b86a35f-c113-4c27-823e-137a0e244f5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:47.490529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2468da22-1c3c-42a4-8bbd-a7151fd738c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:47.493383 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2468da22-1c3c-42a4-8bbd-a7151fd738c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:47.502735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2468da22-1c3c-42a4-8bbd-a7151fd738c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:47.503609 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2468da22-1c3c-42a4-8bbd-a7151fd738c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:47.504458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2468da22-1c3c-42a4-8bbd-a7151fd738c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:47.505715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2468da22-1c3c-42a4-8bbd-a7151fd738c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:47.506786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2468da22-1c3c-42a4-8bbd-a7151fd738c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:47.507352 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2468da22-1c3c-42a4-8bbd-a7151fd738c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:47.547908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2468da22-1c3c-42a4-8bbd-a7151fd738c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062361 Aug 07 01:18:47.548493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 458/886] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:47 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:18:47.568156 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3b86a35f-c113-4c27-823e-137a0e244f5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:47.577774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3b86a35f-c113-4c27-823e-137a0e244f5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.107664 Aug 07 01:18:47.578379 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 429/887] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:47 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:48.567544 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1c00f588-8268-43be-8fba-fd033e271f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:48.571318 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1c00f588-8268-43be-8fba-fd033e271f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:48.583650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c00f588-8268-43be-8fba-fd033e271f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:48.584552 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c00f588-8268-43be-8fba-fd033e271f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:48.585325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c00f588-8268-43be-8fba-fd033e271f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:48.586585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c00f588-8268-43be-8fba-fd033e271f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:48.587422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c00f588-8268-43be-8fba-fd033e271f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:48.588196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c00f588-8268-43be-8fba-fd033e271f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:48.600492 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ed8424cf-2fd8-40e3-b7be-ddabe7a01b15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:48.601195 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ed8424cf-2fd8-40e3-b7be-ddabe7a01b15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:48.609543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ed8424cf-2fd8-40e3-b7be-ddabe7a01b15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:48.610210 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ed8424cf-2fd8-40e3-b7be-ddabe7a01b15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:48.610743 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ed8424cf-2fd8-40e3-b7be-ddabe7a01b15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:48.611577 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ed8424cf-2fd8-40e3-b7be-ddabe7a01b15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:48.612219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ed8424cf-2fd8-40e3-b7be-ddabe7a01b15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:48.612778 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ed8424cf-2fd8-40e3-b7be-ddabe7a01b15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:48.644663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1c00f588-8268-43be-8fba-fd033e271f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.086201 Aug 07 01:18:48.645668 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 459/888] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:48 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:48.697027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ed8424cf-2fd8-40e3-b7be-ddabe7a01b15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:48.706173 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ed8424cf-2fd8-40e3-b7be-ddabe7a01b15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.110375 Aug 07 01:18:48.706920 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 430/889] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:48 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:49.660986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-78548d81-721e-4b74-8626-d20281273476 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:49.663489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-78548d81-721e-4b74-8626-d20281273476 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:49.672193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78548d81-721e-4b74-8626-d20281273476 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:49.672724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78548d81-721e-4b74-8626-d20281273476 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:49.673162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78548d81-721e-4b74-8626-d20281273476 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:49.673894 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78548d81-721e-4b74-8626-d20281273476 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:49.674348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78548d81-721e-4b74-8626-d20281273476 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:49.674907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78548d81-721e-4b74-8626-d20281273476 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:49.714420 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-78548d81-721e-4b74-8626-d20281273476 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057988 Aug 07 01:18:49.715030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 460/890] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:49 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:49.735479 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-58e2e3f5-f144-4f0b-abb3-1c22f2d84d7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:49.736672 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-58e2e3f5-f144-4f0b-abb3-1c22f2d84d7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:49.748779 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-58e2e3f5-f144-4f0b-abb3-1c22f2d84d7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:49.749522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-58e2e3f5-f144-4f0b-abb3-1c22f2d84d7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:49.750348 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-58e2e3f5-f144-4f0b-abb3-1c22f2d84d7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:49.751679 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-58e2e3f5-f144-4f0b-abb3-1c22f2d84d7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:49.752507 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-58e2e3f5-f144-4f0b-abb3-1c22f2d84d7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:49.753344 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-58e2e3f5-f144-4f0b-abb3-1c22f2d84d7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:49.867475 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-58e2e3f5-f144-4f0b-abb3-1c22f2d84d7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:49.881396 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-58e2e3f5-f144-4f0b-abb3-1c22f2d84d7b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.153023 Aug 07 01:18:49.882437 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 431/891] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:49 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:50.732528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bf1e554c-02ee-4853-8ea7-f83da2f8c30d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:50.735764 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bf1e554c-02ee-4853-8ea7-f83da2f8c30d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:50.744794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bf1e554c-02ee-4853-8ea7-f83da2f8c30d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:50.746142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bf1e554c-02ee-4853-8ea7-f83da2f8c30d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:50.746926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bf1e554c-02ee-4853-8ea7-f83da2f8c30d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:50.747921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bf1e554c-02ee-4853-8ea7-f83da2f8c30d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:50.748469 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bf1e554c-02ee-4853-8ea7-f83da2f8c30d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:50.748906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bf1e554c-02ee-4853-8ea7-f83da2f8c30d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:50.811132 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bf1e554c-02ee-4853-8ea7-f83da2f8c30d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.084487 Aug 07 01:18:50.813525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 461/892] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:50 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:50.919296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2978893f-368a-4830-95a0-389d16991935 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:50.920293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2978893f-368a-4830-95a0-389d16991935 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:50.932435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2978893f-368a-4830-95a0-389d16991935 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:50.933877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2978893f-368a-4830-95a0-389d16991935 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:50.934663 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2978893f-368a-4830-95a0-389d16991935 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:50.936028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2978893f-368a-4830-95a0-389d16991935 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:50.936922 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2978893f-368a-4830-95a0-389d16991935 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:50.937707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2978893f-368a-4830-95a0-389d16991935 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:51.046011 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2978893f-368a-4830-95a0-389d16991935 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:51.060621 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2978893f-368a-4830-95a0-389d16991935 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.148024 Aug 07 01:18:51.061688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 432/893] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:50 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:51.831596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4be8d178-4161-4467-9beb-88ea43cef590 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:51.835477 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4be8d178-4161-4467-9beb-88ea43cef590 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:51.848014 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4be8d178-4161-4467-9beb-88ea43cef590 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:51.848986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4be8d178-4161-4467-9beb-88ea43cef590 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:51.849795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4be8d178-4161-4467-9beb-88ea43cef590 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:51.851240 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4be8d178-4161-4467-9beb-88ea43cef590 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:51.852123 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4be8d178-4161-4467-9beb-88ea43cef590 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:51.852963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4be8d178-4161-4467-9beb-88ea43cef590 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:51.915136 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4be8d178-4161-4467-9beb-88ea43cef590 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.091906 Aug 07 01:18:51.916197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 462/894] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:51 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:52.099106 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5bffaf54-8cb5-4f7d-9c3b-9f86c455a67b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:52.100182 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5bffaf54-8cb5-4f7d-9c3b-9f86c455a67b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:52.112800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5bffaf54-8cb5-4f7d-9c3b-9f86c455a67b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:52.113671 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5bffaf54-8cb5-4f7d-9c3b-9f86c455a67b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:52.114454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5bffaf54-8cb5-4f7d-9c3b-9f86c455a67b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:52.115783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5bffaf54-8cb5-4f7d-9c3b-9f86c455a67b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:52.116639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5bffaf54-8cb5-4f7d-9c3b-9f86c455a67b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:52.117419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5bffaf54-8cb5-4f7d-9c3b-9f86c455a67b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:52.222144 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5bffaf54-8cb5-4f7d-9c3b-9f86c455a67b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:52.237194 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5bffaf54-8cb5-4f7d-9c3b-9f86c455a67b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.145547 Aug 07 01:18:52.238249 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 433/895] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:52 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:52.934266 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a1527ebe-ae26-41fc-964d-c7fa898c6593 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:52.938161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a1527ebe-ae26-41fc-964d-c7fa898c6593 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:52.950620 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1527ebe-ae26-41fc-964d-c7fa898c6593 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:52.951738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1527ebe-ae26-41fc-964d-c7fa898c6593 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:52.952553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1527ebe-ae26-41fc-964d-c7fa898c6593 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:52.953870 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1527ebe-ae26-41fc-964d-c7fa898c6593 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:52.954703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1527ebe-ae26-41fc-964d-c7fa898c6593 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:52.955463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1527ebe-ae26-41fc-964d-c7fa898c6593 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:53.006655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:18:53.016302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a1527ebe-ae26-41fc-964d-c7fa898c6593 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.089885 Aug 07 01:18:53.017326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 463/896] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:52 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:53.271912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e2ea7a82-6420-4bd3-886c-7208666b43da tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:53.272911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e2ea7a82-6420-4bd3-886c-7208666b43da tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:53.280913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2ea7a82-6420-4bd3-886c-7208666b43da tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:53.281446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2ea7a82-6420-4bd3-886c-7208666b43da tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:53.282064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2ea7a82-6420-4bd3-886c-7208666b43da tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:53.282575 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2ea7a82-6420-4bd3-886c-7208666b43da tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:53.283361 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2ea7a82-6420-4bd3-886c-7208666b43da tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:53.284554 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2ea7a82-6420-4bd3-886c-7208666b43da tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:53.354230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e2ea7a82-6420-4bd3-886c-7208666b43da tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:53.364898 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e2ea7a82-6420-4bd3-886c-7208666b43da tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097327 Aug 07 01:18:53.365615 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 434/897] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:53 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:54.032403 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b10f7a2b-8d00-44d9-b3e0-ffb6fae4a7e6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:54.034624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b10f7a2b-8d00-44d9-b3e0-ffb6fae4a7e6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:54.041086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b10f7a2b-8d00-44d9-b3e0-ffb6fae4a7e6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:54.041737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b10f7a2b-8d00-44d9-b3e0-ffb6fae4a7e6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:54.042800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b10f7a2b-8d00-44d9-b3e0-ffb6fae4a7e6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:54.043623 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b10f7a2b-8d00-44d9-b3e0-ffb6fae4a7e6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:54.044175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b10f7a2b-8d00-44d9-b3e0-ffb6fae4a7e6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:54.044731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b10f7a2b-8d00-44d9-b3e0-ffb6fae4a7e6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:54.079906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b10f7a2b-8d00-44d9-b3e0-ffb6fae4a7e6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053419 Aug 07 01:18:54.080607 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 464/898] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:54 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:54.387821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b9b85c7f-306a-4e9d-a395-2c7857900b1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:54.388478 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b9b85c7f-306a-4e9d-a395-2c7857900b1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:54.395966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b9b85c7f-306a-4e9d-a395-2c7857900b1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:54.396539 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b9b85c7f-306a-4e9d-a395-2c7857900b1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:54.396991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b9b85c7f-306a-4e9d-a395-2c7857900b1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:54.397976 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b9b85c7f-306a-4e9d-a395-2c7857900b1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:54.398914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b9b85c7f-306a-4e9d-a395-2c7857900b1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:54.398914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b9b85c7f-306a-4e9d-a395-2c7857900b1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:54.470875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b9b85c7f-306a-4e9d-a395-2c7857900b1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:54.479158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b9b85c7f-306a-4e9d-a395-2c7857900b1c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.096113 Aug 07 01:18:54.480345 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 435/899] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:54 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:55.096183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2253694f-6067-4495-8751-1d0a2c118e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:55.099062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2253694f-6067-4495-8751-1d0a2c118e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:55.108284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2253694f-6067-4495-8751-1d0a2c118e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:55.109004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2253694f-6067-4495-8751-1d0a2c118e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:55.109455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2253694f-6067-4495-8751-1d0a2c118e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:55.110280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2253694f-6067-4495-8751-1d0a2c118e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:55.110769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2253694f-6067-4495-8751-1d0a2c118e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:55.111212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2253694f-6067-4495-8751-1d0a2c118e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:55.147477 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2253694f-6067-4495-8751-1d0a2c118e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057012 Aug 07 01:18:55.147911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 465/900] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:55 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:55.504441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-82cc1423-70ad-40bb-9db2-5e4c01b5a317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:55.506245 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-82cc1423-70ad-40bb-9db2-5e4c01b5a317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:55.517799 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-82cc1423-70ad-40bb-9db2-5e4c01b5a317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:55.518653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-82cc1423-70ad-40bb-9db2-5e4c01b5a317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:55.519450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-82cc1423-70ad-40bb-9db2-5e4c01b5a317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:55.520656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-82cc1423-70ad-40bb-9db2-5e4c01b5a317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:55.521513 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-82cc1423-70ad-40bb-9db2-5e4c01b5a317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:55.522248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-82cc1423-70ad-40bb-9db2-5e4c01b5a317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:55.615328 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-82cc1423-70ad-40bb-9db2-5e4c01b5a317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:55.624283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-82cc1423-70ad-40bb-9db2-5e4c01b5a317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.126956 Aug 07 01:18:55.625110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 436/901] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:55 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:56.164298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-caf68a8f-a4b2-4a12-8e0d-9e2ffab196ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:56.168259 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-caf68a8f-a4b2-4a12-8e0d-9e2ffab196ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:56.179672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caf68a8f-a4b2-4a12-8e0d-9e2ffab196ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:56.180212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caf68a8f-a4b2-4a12-8e0d-9e2ffab196ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:56.180736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caf68a8f-a4b2-4a12-8e0d-9e2ffab196ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:56.181517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caf68a8f-a4b2-4a12-8e0d-9e2ffab196ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:56.182020 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caf68a8f-a4b2-4a12-8e0d-9e2ffab196ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:56.182561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caf68a8f-a4b2-4a12-8e0d-9e2ffab196ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:56.220917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-caf68a8f-a4b2-4a12-8e0d-9e2ffab196ef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063969 Aug 07 01:18:56.221593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 466/902] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:56 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:56.646817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6391aa17-cad4-469a-a353-f30fb322ea15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:56.647560 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6391aa17-cad4-469a-a353-f30fb322ea15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:56.657551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6391aa17-cad4-469a-a353-f30fb322ea15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:56.658627 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6391aa17-cad4-469a-a353-f30fb322ea15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:56.659403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6391aa17-cad4-469a-a353-f30fb322ea15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:56.660710 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6391aa17-cad4-469a-a353-f30fb322ea15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:56.661567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6391aa17-cad4-469a-a353-f30fb322ea15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:56.662310 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6391aa17-cad4-469a-a353-f30fb322ea15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:56.765077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6391aa17-cad4-469a-a353-f30fb322ea15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:56.778332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6391aa17-cad4-469a-a353-f30fb322ea15 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.135291 Aug 07 01:18:56.779501 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 437/903] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:56 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:57.234287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6506f094-8e7c-45d6-8068-b579b0565006 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:57.236907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6506f094-8e7c-45d6-8068-b579b0565006 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:57.244533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6506f094-8e7c-45d6-8068-b579b0565006 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:57.245481 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6506f094-8e7c-45d6-8068-b579b0565006 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:57.246600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6506f094-8e7c-45d6-8068-b579b0565006 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:57.247015 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6506f094-8e7c-45d6-8068-b579b0565006 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:57.247493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6506f094-8e7c-45d6-8068-b579b0565006 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:57.247923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6506f094-8e7c-45d6-8068-b579b0565006 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:57.290690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6506f094-8e7c-45d6-8068-b579b0565006 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060865 Aug 07 01:18:57.291289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 467/904] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:57 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:57.803724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4b73daa6-5d7f-4d1e-abe5-9f24e3d4c357 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:57.804442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4b73daa6-5d7f-4d1e-abe5-9f24e3d4c357 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:57.813637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b73daa6-5d7f-4d1e-abe5-9f24e3d4c357 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:57.814296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b73daa6-5d7f-4d1e-abe5-9f24e3d4c357 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:57.814956 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b73daa6-5d7f-4d1e-abe5-9f24e3d4c357 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:57.815883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b73daa6-5d7f-4d1e-abe5-9f24e3d4c357 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:57.816572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b73daa6-5d7f-4d1e-abe5-9f24e3d4c357 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:57.817214 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b73daa6-5d7f-4d1e-abe5-9f24e3d4c357 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:57.897986 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4b73daa6-5d7f-4d1e-abe5-9f24e3d4c357 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:57.912454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4b73daa6-5d7f-4d1e-abe5-9f24e3d4c357 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.112384 Aug 07 01:18:57.913496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 438/905] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:57 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:18:58.304568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3717d245-7444-49bf-a746-2f90a0f17e54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:58.307236 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3717d245-7444-49bf-a746-2f90a0f17e54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:58.314423 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3717d245-7444-49bf-a746-2f90a0f17e54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:58.314921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3717d245-7444-49bf-a746-2f90a0f17e54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:58.315369 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3717d245-7444-49bf-a746-2f90a0f17e54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:58.316103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3717d245-7444-49bf-a746-2f90a0f17e54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:58.316604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3717d245-7444-49bf-a746-2f90a0f17e54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:58.317038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3717d245-7444-49bf-a746-2f90a0f17e54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:58.375280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3717d245-7444-49bf-a746-2f90a0f17e54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.075340 Aug 07 01:18:58.376399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 468/906] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:58 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:18:58.947582 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bae17095-58b4-45f6-b51d-7e64cb41dee6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:58.948740 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bae17095-58b4-45f6-b51d-7e64cb41dee6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:58.955868 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bae17095-58b4-45f6-b51d-7e64cb41dee6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:58.956428 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bae17095-58b4-45f6-b51d-7e64cb41dee6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:58.956933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bae17095-58b4-45f6-b51d-7e64cb41dee6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:58.957645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bae17095-58b4-45f6-b51d-7e64cb41dee6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:58.958124 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bae17095-58b4-45f6-b51d-7e64cb41dee6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:58.958542 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bae17095-58b4-45f6-b51d-7e64cb41dee6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:58.963188 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:18:59.034707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bae17095-58b4-45f6-b51d-7e64cb41dee6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:18:59.046781 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bae17095-58b4-45f6-b51d-7e64cb41dee6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.103745 Aug 07 01:18:59.047448 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 439/907] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:18:58 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:18:59.392741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f8a6bcae-7ffd-4abf-b268-1a84beb4b72d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:18:59.395259 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f8a6bcae-7ffd-4abf-b268-1a84beb4b72d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:18:59.403441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8a6bcae-7ffd-4abf-b268-1a84beb4b72d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:59.404077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8a6bcae-7ffd-4abf-b268-1a84beb4b72d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:59.404750 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8a6bcae-7ffd-4abf-b268-1a84beb4b72d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:59.405980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8a6bcae-7ffd-4abf-b268-1a84beb4b72d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:18:59.406613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8a6bcae-7ffd-4abf-b268-1a84beb4b72d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:18:59.407172 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8a6bcae-7ffd-4abf-b268-1a84beb4b72d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:18:59.444034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f8a6bcae-7ffd-4abf-b268-1a84beb4b72d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055596 Aug 07 01:18:59.444982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 469/908] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:18:59 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:00.068755 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-09b9c11e-e3a7-4e42-84ec-f280ec8764f3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:00.069413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-09b9c11e-e3a7-4e42-84ec-f280ec8764f3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:00.077270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09b9c11e-e3a7-4e42-84ec-f280ec8764f3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:00.077809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09b9c11e-e3a7-4e42-84ec-f280ec8764f3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:00.078343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09b9c11e-e3a7-4e42-84ec-f280ec8764f3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:00.079125 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09b9c11e-e3a7-4e42-84ec-f280ec8764f3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:00.079633 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09b9c11e-e3a7-4e42-84ec-f280ec8764f3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:00.080113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09b9c11e-e3a7-4e42-84ec-f280ec8764f3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:00.155443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-09b9c11e-e3a7-4e42-84ec-f280ec8764f3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:00.168862 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-09b9c11e-e3a7-4e42-84ec-f280ec8764f3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.103876 Aug 07 01:19:00.170160 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 440/909] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:00 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:00.461319 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e97d6c7c-546d-4242-bf34-8220fb3387b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:00.465985 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e97d6c7c-546d-4242-bf34-8220fb3387b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:00.466374 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=56 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:19:00.477457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e97d6c7c-546d-4242-bf34-8220fb3387b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:00.478364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e97d6c7c-546d-4242-bf34-8220fb3387b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:00.479083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e97d6c7c-546d-4242-bf34-8220fb3387b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:00.480982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e97d6c7c-546d-4242-bf34-8220fb3387b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:00.481666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e97d6c7c-546d-4242-bf34-8220fb3387b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:00.482213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e97d6c7c-546d-4242-bf34-8220fb3387b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:00.521592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e97d6c7c-546d-4242-bf34-8220fb3387b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064426 Aug 07 01:19:00.522204 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 470/910] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:00 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:01.193575 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8972386c-d801-47c7-a54a-533fd4a7f498 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:01.194233 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8972386c-d801-47c7-a54a-533fd4a7f498 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:01.202573 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8972386c-d801-47c7-a54a-533fd4a7f498 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:01.203066 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8972386c-d801-47c7-a54a-533fd4a7f498 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:01.203545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8972386c-d801-47c7-a54a-533fd4a7f498 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:01.204230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8972386c-d801-47c7-a54a-533fd4a7f498 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:01.204720 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8972386c-d801-47c7-a54a-533fd4a7f498 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:01.205155 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8972386c-d801-47c7-a54a-533fd4a7f498 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:01.277310 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8972386c-d801-47c7-a54a-533fd4a7f498 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:01.285848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8972386c-d801-47c7-a54a-533fd4a7f498 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097319 Aug 07 01:19:01.286438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 441/911] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:01 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:01.534418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3cadad3d-c81a-4c89-b0d8-55b17575d158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:01.536707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3cadad3d-c81a-4c89-b0d8-55b17575d158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:01.544035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3cadad3d-c81a-4c89-b0d8-55b17575d158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:01.544565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3cadad3d-c81a-4c89-b0d8-55b17575d158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:01.545011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3cadad3d-c81a-4c89-b0d8-55b17575d158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:01.545742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3cadad3d-c81a-4c89-b0d8-55b17575d158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:01.546211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3cadad3d-c81a-4c89-b0d8-55b17575d158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:01.546643 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3cadad3d-c81a-4c89-b0d8-55b17575d158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:01.579843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3cadad3d-c81a-4c89-b0d8-55b17575d158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049518 Aug 07 01:19:01.580529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 471/912] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:01 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:02.308587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-830734b4-e66d-473c-911e-6f030d5f9046 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:02.309386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-830734b4-e66d-473c-911e-6f030d5f9046 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:02.312250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=54 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:19:02.317887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-830734b4-e66d-473c-911e-6f030d5f9046 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:02.318556 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-830734b4-e66d-473c-911e-6f030d5f9046 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:02.319268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-830734b4-e66d-473c-911e-6f030d5f9046 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:02.319998 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-830734b4-e66d-473c-911e-6f030d5f9046 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:02.320555 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-830734b4-e66d-473c-911e-6f030d5f9046 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:02.320969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-830734b4-e66d-473c-911e-6f030d5f9046 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:02.392911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-830734b4-e66d-473c-911e-6f030d5f9046 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:02.401785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-830734b4-e66d-473c-911e-6f030d5f9046 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.098284 Aug 07 01:19:02.402639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 442/913] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:02 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:02.595843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2810b542-6ae0-4b9d-801d-bc8f94528305 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:02.599191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2810b542-6ae0-4b9d-801d-bc8f94528305 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:02.610711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2810b542-6ae0-4b9d-801d-bc8f94528305 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:02.611571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2810b542-6ae0-4b9d-801d-bc8f94528305 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:02.612392 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2810b542-6ae0-4b9d-801d-bc8f94528305 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:02.613681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2810b542-6ae0-4b9d-801d-bc8f94528305 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:02.614527 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2810b542-6ae0-4b9d-801d-bc8f94528305 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:02.615257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2810b542-6ae0-4b9d-801d-bc8f94528305 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:02.658358 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2810b542-6ae0-4b9d-801d-bc8f94528305 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069552 Aug 07 01:19:02.659078 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 472/914] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:02 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:03.424587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ce66743f-11a4-4255-93f8-9b2cf30e092a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:03.425415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ce66743f-11a4-4255-93f8-9b2cf30e092a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:03.434406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce66743f-11a4-4255-93f8-9b2cf30e092a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:03.434969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce66743f-11a4-4255-93f8-9b2cf30e092a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:03.435521 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce66743f-11a4-4255-93f8-9b2cf30e092a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:03.436458 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce66743f-11a4-4255-93f8-9b2cf30e092a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:03.436915 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce66743f-11a4-4255-93f8-9b2cf30e092a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:03.437396 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce66743f-11a4-4255-93f8-9b2cf30e092a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:03.521352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ce66743f-11a4-4255-93f8-9b2cf30e092a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:03.535571 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ce66743f-11a4-4255-93f8-9b2cf30e092a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.115367 Aug 07 01:19:03.536677 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 443/915] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:03 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:03.671994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-45f83ef7-27c6-43f7-ad7a-6124f7c5c15a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:03.674654 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-45f83ef7-27c6-43f7-ad7a-6124f7c5c15a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:03.683108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45f83ef7-27c6-43f7-ad7a-6124f7c5c15a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:03.683609 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45f83ef7-27c6-43f7-ad7a-6124f7c5c15a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:03.684057 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45f83ef7-27c6-43f7-ad7a-6124f7c5c15a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:03.684826 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45f83ef7-27c6-43f7-ad7a-6124f7c5c15a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:03.685280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45f83ef7-27c6-43f7-ad7a-6124f7c5c15a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:03.685710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-45f83ef7-27c6-43f7-ad7a-6124f7c5c15a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:03.723298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-45f83ef7-27c6-43f7-ad7a-6124f7c5c15a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055352 Aug 07 01:19:03.723903 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 473/916] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:03 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:04.565964 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c6169139-5979-4d0d-90d1-79ce8bc47b8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:04.567146 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c6169139-5979-4d0d-90d1-79ce8bc47b8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:04.580140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6169139-5979-4d0d-90d1-79ce8bc47b8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:04.581138 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6169139-5979-4d0d-90d1-79ce8bc47b8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:04.582092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6169139-5979-4d0d-90d1-79ce8bc47b8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:04.583499 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6169139-5979-4d0d-90d1-79ce8bc47b8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:04.584553 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6169139-5979-4d0d-90d1-79ce8bc47b8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:04.585381 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6169139-5979-4d0d-90d1-79ce8bc47b8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:04.671794 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c6169139-5979-4d0d-90d1-79ce8bc47b8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:04.679831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c6169139-5979-4d0d-90d1-79ce8bc47b8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.122522 Aug 07 01:19:04.680822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 444/917] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:04 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:04.740155 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ef11a49b-efac-4fbf-9e60-7feef6a75d63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:04.742505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ef11a49b-efac-4fbf-9e60-7feef6a75d63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:04.750294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef11a49b-efac-4fbf-9e60-7feef6a75d63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:04.750840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef11a49b-efac-4fbf-9e60-7feef6a75d63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:04.751284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef11a49b-efac-4fbf-9e60-7feef6a75d63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:04.752191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef11a49b-efac-4fbf-9e60-7feef6a75d63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:04.752617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef11a49b-efac-4fbf-9e60-7feef6a75d63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:04.753196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef11a49b-efac-4fbf-9e60-7feef6a75d63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:04.787472 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ef11a49b-efac-4fbf-9e60-7feef6a75d63 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051624 Aug 07 01:19:04.788144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 474/918] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:04 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:05.703126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ae390c6b-1683-4126-bef2-42f323b092e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:05.703959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ae390c6b-1683-4126-bef2-42f323b092e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:05.711704 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae390c6b-1683-4126-bef2-42f323b092e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:05.712290 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae390c6b-1683-4126-bef2-42f323b092e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:05.712820 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae390c6b-1683-4126-bef2-42f323b092e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:05.713723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae390c6b-1683-4126-bef2-42f323b092e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:05.714241 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae390c6b-1683-4126-bef2-42f323b092e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:05.714721 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae390c6b-1683-4126-bef2-42f323b092e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:05.802004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ae390c6b-1683-4126-bef2-42f323b092e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:05.802532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a0217e93-d66e-4b9b-a8e2-d5b0fcfe1ba2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:05.805120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a0217e93-d66e-4b9b-a8e2-d5b0fcfe1ba2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:05.812596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0217e93-d66e-4b9b-a8e2-d5b0fcfe1ba2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:05.813003 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0217e93-d66e-4b9b-a8e2-d5b0fcfe1ba2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:05.813577 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0217e93-d66e-4b9b-a8e2-d5b0fcfe1ba2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:05.814341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0217e93-d66e-4b9b-a8e2-d5b0fcfe1ba2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:05.814952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0217e93-d66e-4b9b-a8e2-d5b0fcfe1ba2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:05.815520 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0217e93-d66e-4b9b-a8e2-d5b0fcfe1ba2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:05.815644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ae390c6b-1683-4126-bef2-42f323b092e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.117535 Aug 07 01:19:05.816621 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 445/919] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:05 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:05.856099 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a0217e93-d66e-4b9b-a8e2-d5b0fcfe1ba2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058272 Aug 07 01:19:05.856740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 475/920] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:05 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:06.852551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ff0a0cc3-e97a-4e3f-8601-dff51fc0a556 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:06.853733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ff0a0cc3-e97a-4e3f-8601-dff51fc0a556 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:06.866459 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff0a0cc3-e97a-4e3f-8601-dff51fc0a556 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:06.866969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff0a0cc3-e97a-4e3f-8601-dff51fc0a556 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:06.867519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff0a0cc3-e97a-4e3f-8601-dff51fc0a556 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:06.868220 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff0a0cc3-e97a-4e3f-8601-dff51fc0a556 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:06.868823 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff0a0cc3-e97a-4e3f-8601-dff51fc0a556 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:06.869314 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff0a0cc3-e97a-4e3f-8601-dff51fc0a556 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:06.874458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6772404e-b3a7-4c45-9e87-618bb12a013b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:06.878477 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6772404e-b3a7-4c45-9e87-618bb12a013b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:06.886081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6772404e-b3a7-4c45-9e87-618bb12a013b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:06.886591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6772404e-b3a7-4c45-9e87-618bb12a013b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:06.887044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6772404e-b3a7-4c45-9e87-618bb12a013b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:06.887792 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6772404e-b3a7-4c45-9e87-618bb12a013b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:06.888286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6772404e-b3a7-4c45-9e87-618bb12a013b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:06.888720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6772404e-b3a7-4c45-9e87-618bb12a013b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:06.950199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6772404e-b3a7-4c45-9e87-618bb12a013b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.083260 Aug 07 01:19:06.955478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 476/921] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:06 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:06.978779 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ff0a0cc3-e97a-4e3f-8601-dff51fc0a556 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:06.993642 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ff0a0cc3-e97a-4e3f-8601-dff51fc0a556 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.147067 Aug 07 01:19:06.996520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 446/922] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:06 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:07.964737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:19:07.966229 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:19:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments (ip 10.222.0.26) !!! Aug 07 01:19:07.970800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-62ca4eb0-6d00-4db6-87d3-af8ed5fbbb9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:07.973962 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-62ca4eb0-6d00-4db6-87d3-af8ed5fbbb9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:07.984848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-62ca4eb0-6d00-4db6-87d3-af8ed5fbbb9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:07.985479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-62ca4eb0-6d00-4db6-87d3-af8ed5fbbb9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:07.986002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-62ca4eb0-6d00-4db6-87d3-af8ed5fbbb9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:07.986826 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-62ca4eb0-6d00-4db6-87d3-af8ed5fbbb9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:07.988291 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-62ca4eb0-6d00-4db6-87d3-af8ed5fbbb9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:07.988291 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-62ca4eb0-6d00-4db6-87d3-af8ed5fbbb9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:08.024829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-62ca4eb0-6d00-4db6-87d3-af8ed5fbbb9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058755 Aug 07 01:19:08.025428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 477/923] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:07 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:08.035494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-34ce2660-0479-450d-a4f1-10bf5093c6b5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:08.036746 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-34ce2660-0479-450d-a4f1-10bf5093c6b5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:08.048206 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34ce2660-0479-450d-a4f1-10bf5093c6b5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:08.049103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34ce2660-0479-450d-a4f1-10bf5093c6b5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:08.049895 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34ce2660-0479-450d-a4f1-10bf5093c6b5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:08.051223 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34ce2660-0479-450d-a4f1-10bf5093c6b5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:08.052003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34ce2660-0479-450d-a4f1-10bf5093c6b5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:08.052805 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34ce2660-0479-450d-a4f1-10bf5093c6b5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:08.150606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-34ce2660-0479-450d-a4f1-10bf5093c6b5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:08.160498 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-34ce2660-0479-450d-a4f1-10bf5093c6b5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.131936 Aug 07 01:19:08.172280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 447/924] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:08 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:09.039267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4bd57d86-112b-42cc-bd6d-cc84c583ec6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:09.041981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4bd57d86-112b-42cc-bd6d-cc84c583ec6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:09.049578 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4bd57d86-112b-42cc-bd6d-cc84c583ec6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:09.050075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4bd57d86-112b-42cc-bd6d-cc84c583ec6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:09.050560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4bd57d86-112b-42cc-bd6d-cc84c583ec6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:09.051404 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4bd57d86-112b-42cc-bd6d-cc84c583ec6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:09.051918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4bd57d86-112b-42cc-bd6d-cc84c583ec6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:09.052695 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4bd57d86-112b-42cc-bd6d-cc84c583ec6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:09.110527 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4bd57d86-112b-42cc-bd6d-cc84c583ec6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.075940 Aug 07 01:19:09.111135 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 478/925] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:09 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:09.190615 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-845df6c2-02d5-43af-9906-c7f994848a70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:09.191475 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-845df6c2-02d5-43af-9906-c7f994848a70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:09.199654 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-845df6c2-02d5-43af-9906-c7f994848a70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:09.200267 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-845df6c2-02d5-43af-9906-c7f994848a70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:09.200803 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-845df6c2-02d5-43af-9906-c7f994848a70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:09.202028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-845df6c2-02d5-43af-9906-c7f994848a70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:09.202618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-845df6c2-02d5-43af-9906-c7f994848a70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:09.203132 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-845df6c2-02d5-43af-9906-c7f994848a70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:09.287362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-845df6c2-02d5-43af-9906-c7f994848a70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:09.296189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-845df6c2-02d5-43af-9906-c7f994848a70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.109530 Aug 07 01:19:09.296986 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 448/926] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:09 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:10.125154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1a71a49e-1ef4-48fb-bb05-645af82b194d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:10.128034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1a71a49e-1ef4-48fb-bb05-645af82b194d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:10.136606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a71a49e-1ef4-48fb-bb05-645af82b194d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:10.137107 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a71a49e-1ef4-48fb-bb05-645af82b194d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:10.137556 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a71a49e-1ef4-48fb-bb05-645af82b194d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:10.138308 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a71a49e-1ef4-48fb-bb05-645af82b194d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:10.138773 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a71a49e-1ef4-48fb-bb05-645af82b194d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:10.139208 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a71a49e-1ef4-48fb-bb05-645af82b194d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:10.178725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1a71a49e-1ef4-48fb-bb05-645af82b194d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059227 Aug 07 01:19:10.179319 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 479/927] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:10 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:10.319088 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d4b2594d-8b29-42b3-b6cd-5e6338590002 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:10.319743 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d4b2594d-8b29-42b3-b6cd-5e6338590002 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:10.327763 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d4b2594d-8b29-42b3-b6cd-5e6338590002 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:10.328280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d4b2594d-8b29-42b3-b6cd-5e6338590002 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:10.328797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d4b2594d-8b29-42b3-b6cd-5e6338590002 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:10.329938 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d4b2594d-8b29-42b3-b6cd-5e6338590002 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:10.329938 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d4b2594d-8b29-42b3-b6cd-5e6338590002 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:10.330354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d4b2594d-8b29-42b3-b6cd-5e6338590002 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:10.406384 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d4b2594d-8b29-42b3-b6cd-5e6338590002 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:10.414915 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d4b2594d-8b29-42b3-b6cd-5e6338590002 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100041 Aug 07 01:19:10.415531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 449/928] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:10 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:11.191908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d1b82774-599d-463c-ac9d-02f813ec9d40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:11.194215 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d1b82774-599d-463c-ac9d-02f813ec9d40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:11.203325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1b82774-599d-463c-ac9d-02f813ec9d40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:11.203841 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1b82774-599d-463c-ac9d-02f813ec9d40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:11.204335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1b82774-599d-463c-ac9d-02f813ec9d40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:11.205282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1b82774-599d-463c-ac9d-02f813ec9d40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:11.205755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1b82774-599d-463c-ac9d-02f813ec9d40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:11.206295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1b82774-599d-463c-ac9d-02f813ec9d40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:11.227861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:19:11.244100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d1b82774-599d-463c-ac9d-02f813ec9d40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056338 Aug 07 01:19:11.244863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 480/929] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:11 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:11.437655 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-173b6269-a760-4694-94d8-d32f53e534cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:11.438681 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-173b6269-a760-4694-94d8-d32f53e534cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:11.446543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-173b6269-a760-4694-94d8-d32f53e534cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:11.447003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-173b6269-a760-4694-94d8-d32f53e534cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:11.447491 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-173b6269-a760-4694-94d8-d32f53e534cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:11.448401 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-173b6269-a760-4694-94d8-d32f53e534cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:11.448945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-173b6269-a760-4694-94d8-d32f53e534cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:11.449426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-173b6269-a760-4694-94d8-d32f53e534cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:11.521147 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-173b6269-a760-4694-94d8-d32f53e534cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:11.530716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-173b6269-a760-4694-94d8-d32f53e534cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097942 Aug 07 01:19:11.531450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 450/930] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:11 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:12.261882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-735ebf8e-0043-4205-8dd7-8f46c756732f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:12.265901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-735ebf8e-0043-4205-8dd7-8f46c756732f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:12.277902 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735ebf8e-0043-4205-8dd7-8f46c756732f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:12.278810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735ebf8e-0043-4205-8dd7-8f46c756732f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:12.279594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735ebf8e-0043-4205-8dd7-8f46c756732f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:12.280947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735ebf8e-0043-4205-8dd7-8f46c756732f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:12.281800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735ebf8e-0043-4205-8dd7-8f46c756732f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:12.282587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735ebf8e-0043-4205-8dd7-8f46c756732f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:12.342442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-735ebf8e-0043-4205-8dd7-8f46c756732f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.087967 Aug 07 01:19:12.343486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 481/931] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:12 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:12.558261 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c221b652-3e0d-44e5-b1e2-33cbc29ea955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:12.559400 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c221b652-3e0d-44e5-b1e2-33cbc29ea955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:12.570545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c221b652-3e0d-44e5-b1e2-33cbc29ea955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:12.571425 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c221b652-3e0d-44e5-b1e2-33cbc29ea955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:12.572310 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c221b652-3e0d-44e5-b1e2-33cbc29ea955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:12.573579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c221b652-3e0d-44e5-b1e2-33cbc29ea955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:12.574397 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c221b652-3e0d-44e5-b1e2-33cbc29ea955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:12.575179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c221b652-3e0d-44e5-b1e2-33cbc29ea955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:12.687454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c221b652-3e0d-44e5-b1e2-33cbc29ea955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:12.696742 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c221b652-3e0d-44e5-b1e2-33cbc29ea955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.146753 Aug 07 01:19:12.698143 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 451/932] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:12 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:13.358928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a17648a3-fe85-4df1-9371-33f835c0aad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:13.361698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a17648a3-fe85-4df1-9371-33f835c0aad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:13.369114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a17648a3-fe85-4df1-9371-33f835c0aad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:13.369713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a17648a3-fe85-4df1-9371-33f835c0aad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:13.370159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a17648a3-fe85-4df1-9371-33f835c0aad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:13.370883 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a17648a3-fe85-4df1-9371-33f835c0aad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:13.371348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a17648a3-fe85-4df1-9371-33f835c0aad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:13.371769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a17648a3-fe85-4df1-9371-33f835c0aad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:13.409039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a17648a3-fe85-4df1-9371-33f835c0aad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055148 Aug 07 01:19:13.409741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 482/933] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:13 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:13.731603 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4d705433-07dc-49f9-8da6-8bd12fb65b4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:13.732206 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4d705433-07dc-49f9-8da6-8bd12fb65b4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:13.739775 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d705433-07dc-49f9-8da6-8bd12fb65b4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:13.740462 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d705433-07dc-49f9-8da6-8bd12fb65b4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:13.740973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d705433-07dc-49f9-8da6-8bd12fb65b4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:13.741777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d705433-07dc-49f9-8da6-8bd12fb65b4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:13.742301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d705433-07dc-49f9-8da6-8bd12fb65b4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:13.742825 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d705433-07dc-49f9-8da6-8bd12fb65b4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:13.825781 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4d705433-07dc-49f9-8da6-8bd12fb65b4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:13.835849 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4d705433-07dc-49f9-8da6-8bd12fb65b4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.108151 Aug 07 01:19:13.836883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 452/934] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:13 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:14.422861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cfd2fcea-62b5-497f-ab76-c2014035be75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:14.425324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cfd2fcea-62b5-497f-ab76-c2014035be75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:14.433682 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cfd2fcea-62b5-497f-ab76-c2014035be75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:14.434300 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cfd2fcea-62b5-497f-ab76-c2014035be75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:14.434801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cfd2fcea-62b5-497f-ab76-c2014035be75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:14.435799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cfd2fcea-62b5-497f-ab76-c2014035be75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:14.436376 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cfd2fcea-62b5-497f-ab76-c2014035be75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:14.436967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cfd2fcea-62b5-497f-ab76-c2014035be75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:14.471474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cfd2fcea-62b5-497f-ab76-c2014035be75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053159 Aug 07 01:19:14.472077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 483/935] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:14 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:14.870486 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fe6dd3b2-3121-4dc3-8b8e-df8152b11e24 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:14.871373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fe6dd3b2-3121-4dc3-8b8e-df8152b11e24 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:14.881652 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe6dd3b2-3121-4dc3-8b8e-df8152b11e24 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:14.882157 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe6dd3b2-3121-4dc3-8b8e-df8152b11e24 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:14.882607 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe6dd3b2-3121-4dc3-8b8e-df8152b11e24 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:14.883393 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe6dd3b2-3121-4dc3-8b8e-df8152b11e24 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:14.883958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe6dd3b2-3121-4dc3-8b8e-df8152b11e24 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:14.884509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe6dd3b2-3121-4dc3-8b8e-df8152b11e24 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:14.970336 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fe6dd3b2-3121-4dc3-8b8e-df8152b11e24 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:14.979133 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fe6dd3b2-3121-4dc3-8b8e-df8152b11e24 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.112595 Aug 07 01:19:14.979730 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 453/936] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:14 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:15.490306 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f34c97ee-92f3-49ca-8c6c-8639b504c594 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:15.494839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f34c97ee-92f3-49ca-8c6c-8639b504c594 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:15.510279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f34c97ee-92f3-49ca-8c6c-8639b504c594 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:15.511152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f34c97ee-92f3-49ca-8c6c-8639b504c594 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:15.511974 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f34c97ee-92f3-49ca-8c6c-8639b504c594 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:15.513327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f34c97ee-92f3-49ca-8c6c-8639b504c594 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:15.514162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f34c97ee-92f3-49ca-8c6c-8639b504c594 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:15.514910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f34c97ee-92f3-49ca-8c6c-8639b504c594 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:15.577882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f34c97ee-92f3-49ca-8c6c-8639b504c594 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.095349 Aug 07 01:19:15.578854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 484/937] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:15 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:16.002701 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1d92ed56-600e-4077-b7a6-56f75223ed2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:16.003520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1d92ed56-600e-4077-b7a6-56f75223ed2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:16.012279 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d92ed56-600e-4077-b7a6-56f75223ed2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:16.012918 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d92ed56-600e-4077-b7a6-56f75223ed2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:16.013540 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d92ed56-600e-4077-b7a6-56f75223ed2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:16.014457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d92ed56-600e-4077-b7a6-56f75223ed2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:16.015087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d92ed56-600e-4077-b7a6-56f75223ed2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:16.015602 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1d92ed56-600e-4077-b7a6-56f75223ed2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:16.088923 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1d92ed56-600e-4077-b7a6-56f75223ed2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:16.102590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1d92ed56-600e-4077-b7a6-56f75223ed2d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.103838 Aug 07 01:19:16.103642 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 454/938] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:15 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:16.597505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6ac70605-482e-4b46-9be9-9438803aaf0c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:16.600730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6ac70605-482e-4b46-9be9-9438803aaf0c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:16.609012 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ac70605-482e-4b46-9be9-9438803aaf0c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:16.609788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ac70605-482e-4b46-9be9-9438803aaf0c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:16.610247 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ac70605-482e-4b46-9be9-9438803aaf0c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:16.611050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ac70605-482e-4b46-9be9-9438803aaf0c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:16.611595 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ac70605-482e-4b46-9be9-9438803aaf0c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:16.612290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ac70605-482e-4b46-9be9-9438803aaf0c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:16.656641 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6ac70605-482e-4b46-9be9-9438803aaf0c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064074 Aug 07 01:19:16.657697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 485/939] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:16 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:17.137274 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-221889ac-9ce2-4a8f-829f-1b9f9af832ec tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:17.138007 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-221889ac-9ce2-4a8f-829f-1b9f9af832ec tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:17.145659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-221889ac-9ce2-4a8f-829f-1b9f9af832ec tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:17.145968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-221889ac-9ce2-4a8f-829f-1b9f9af832ec tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:17.146410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-221889ac-9ce2-4a8f-829f-1b9f9af832ec tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:17.147119 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-221889ac-9ce2-4a8f-829f-1b9f9af832ec tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:17.147574 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-221889ac-9ce2-4a8f-829f-1b9f9af832ec tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:17.147995 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-221889ac-9ce2-4a8f-829f-1b9f9af832ec tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:17.221060 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-221889ac-9ce2-4a8f-829f-1b9f9af832ec tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:17.225133 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:19:17.231819 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-221889ac-9ce2-4a8f-829f-1b9f9af832ec tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.099243 Aug 07 01:19:17.232561 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 455/940] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:17 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:17.673693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0765149c-e060-4aab-a402-4c3b07c97257 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:17.675932 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0765149c-e060-4aab-a402-4c3b07c97257 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:17.684399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0765149c-e060-4aab-a402-4c3b07c97257 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:17.684903 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0765149c-e060-4aab-a402-4c3b07c97257 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:17.685364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0765149c-e060-4aab-a402-4c3b07c97257 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:17.686083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0765149c-e060-4aab-a402-4c3b07c97257 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:17.686566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0765149c-e060-4aab-a402-4c3b07c97257 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:17.687008 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0765149c-e060-4aab-a402-4c3b07c97257 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:17.726437 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0765149c-e060-4aab-a402-4c3b07c97257 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056826 Aug 07 01:19:17.727143 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 486/941] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:17 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:18.256127 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-776f8347-3571-435c-b7e1-12f309978254 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:18.257279 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-776f8347-3571-435c-b7e1-12f309978254 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:18.265186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-776f8347-3571-435c-b7e1-12f309978254 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:18.265808 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-776f8347-3571-435c-b7e1-12f309978254 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:18.266267 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-776f8347-3571-435c-b7e1-12f309978254 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:18.267026 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-776f8347-3571-435c-b7e1-12f309978254 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:18.267499 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-776f8347-3571-435c-b7e1-12f309978254 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:18.267946 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-776f8347-3571-435c-b7e1-12f309978254 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:18.344796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-776f8347-3571-435c-b7e1-12f309978254 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:18.353619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-776f8347-3571-435c-b7e1-12f309978254 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.102989 Aug 07 01:19:18.354340 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 456/942] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:18 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:18.744839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-27f1d0ed-36b8-45f5-8405-f931982b0fcd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:18.748556 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-27f1d0ed-36b8-45f5-8405-f931982b0fcd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:18.758753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27f1d0ed-36b8-45f5-8405-f931982b0fcd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:18.759276 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27f1d0ed-36b8-45f5-8405-f931982b0fcd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:18.759835 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27f1d0ed-36b8-45f5-8405-f931982b0fcd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:18.760717 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27f1d0ed-36b8-45f5-8405-f931982b0fcd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:18.761228 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27f1d0ed-36b8-45f5-8405-f931982b0fcd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:18.761710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27f1d0ed-36b8-45f5-8405-f931982b0fcd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:18.801952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-27f1d0ed-36b8-45f5-8405-f931982b0fcd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064923 Aug 07 01:19:18.802853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 487/943] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:18 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:19.387124 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0e8ba840-037d-43bd-a924-771df1d24446 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:19.388015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0e8ba840-037d-43bd-a924-771df1d24446 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:19.395602 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e8ba840-037d-43bd-a924-771df1d24446 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:19.396096 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e8ba840-037d-43bd-a924-771df1d24446 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:19.396643 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e8ba840-037d-43bd-a924-771df1d24446 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:19.397413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e8ba840-037d-43bd-a924-771df1d24446 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:19.397878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e8ba840-037d-43bd-a924-771df1d24446 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:19.398310 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e8ba840-037d-43bd-a924-771df1d24446 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:19.475050 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0e8ba840-037d-43bd-a924-771df1d24446 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:19.489125 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0e8ba840-037d-43bd-a924-771df1d24446 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.105599 Aug 07 01:19:19.490192 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 457/944] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:19 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:19.816448 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b8b3c538-c1b2-4a56-aeb1-b9ee647716f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:19.818680 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b8b3c538-c1b2-4a56-aeb1-b9ee647716f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:19.826585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b8b3c538-c1b2-4a56-aeb1-b9ee647716f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:19.827072 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b8b3c538-c1b2-4a56-aeb1-b9ee647716f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:19.827512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b8b3c538-c1b2-4a56-aeb1-b9ee647716f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:19.828259 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b8b3c538-c1b2-4a56-aeb1-b9ee647716f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:19.828721 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b8b3c538-c1b2-4a56-aeb1-b9ee647716f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:19.829154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b8b3c538-c1b2-4a56-aeb1-b9ee647716f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:19.866598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b8b3c538-c1b2-4a56-aeb1-b9ee647716f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054158 Aug 07 01:19:19.867301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 488/945] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:19 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:20.523827 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4305c5e8-0558-4061-ae36-6b5319a05cba tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:20.524764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4305c5e8-0558-4061-ae36-6b5319a05cba tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:20.532204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4305c5e8-0558-4061-ae36-6b5319a05cba tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:20.532804 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4305c5e8-0558-4061-ae36-6b5319a05cba tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:20.533410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4305c5e8-0558-4061-ae36-6b5319a05cba tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:20.534121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4305c5e8-0558-4061-ae36-6b5319a05cba tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:20.534631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4305c5e8-0558-4061-ae36-6b5319a05cba tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:20.535097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4305c5e8-0558-4061-ae36-6b5319a05cba tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:20.606711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4305c5e8-0558-4061-ae36-6b5319a05cba tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:20.615353 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4305c5e8-0558-4061-ae36-6b5319a05cba tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.095362 Aug 07 01:19:20.615956 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 458/946] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:20 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:20.881789 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f56fbed4-eada-44a3-9f95-810f03702eff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:20.884331 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f56fbed4-eada-44a3-9f95-810f03702eff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:20.891495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f56fbed4-eada-44a3-9f95-810f03702eff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:20.892040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f56fbed4-eada-44a3-9f95-810f03702eff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:20.892573 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f56fbed4-eada-44a3-9f95-810f03702eff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:20.893435 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f56fbed4-eada-44a3-9f95-810f03702eff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:20.893934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f56fbed4-eada-44a3-9f95-810f03702eff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:20.894574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f56fbed4-eada-44a3-9f95-810f03702eff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:20.933915 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f56fbed4-eada-44a3-9f95-810f03702eff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057232 Aug 07 01:19:20.934766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 489/947] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:20 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:21.641684 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-69e024c1-2255-4f93-ab20-0ce25fa879f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:21.642689 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-69e024c1-2255-4f93-ab20-0ce25fa879f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:21.655469 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69e024c1-2255-4f93-ab20-0ce25fa879f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:21.656274 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69e024c1-2255-4f93-ab20-0ce25fa879f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:21.656953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69e024c1-2255-4f93-ab20-0ce25fa879f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:21.657738 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69e024c1-2255-4f93-ab20-0ce25fa879f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:21.658398 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69e024c1-2255-4f93-ab20-0ce25fa879f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:21.658840 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69e024c1-2255-4f93-ab20-0ce25fa879f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:21.745275 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-69e024c1-2255-4f93-ab20-0ce25fa879f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:21.758124 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-69e024c1-2255-4f93-ab20-0ce25fa879f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.124178 Aug 07 01:19:21.759145 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 459/948] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:21 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:21.953711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a4440c42-573f-4cc9-9c6f-37248e69134e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:21.957540 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a4440c42-573f-4cc9-9c6f-37248e69134e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:21.969857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4440c42-573f-4cc9-9c6f-37248e69134e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:21.970709 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4440c42-573f-4cc9-9c6f-37248e69134e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:21.971470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4440c42-573f-4cc9-9c6f-37248e69134e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:21.972805 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4440c42-573f-4cc9-9c6f-37248e69134e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:21.973645 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4440c42-573f-4cc9-9c6f-37248e69134e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:21.974443 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4440c42-573f-4cc9-9c6f-37248e69134e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:22.032660 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a4440c42-573f-4cc9-9c6f-37248e69134e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.087420 Aug 07 01:19:22.033787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 490/949] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:21 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:22.789843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3f0a2a06-d488-4e2c-bd19-c38bbfaf4f6c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:22.790584 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3f0a2a06-d488-4e2c-bd19-c38bbfaf4f6c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:22.797730 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3f0a2a06-d488-4e2c-bd19-c38bbfaf4f6c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:22.798220 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3f0a2a06-d488-4e2c-bd19-c38bbfaf4f6c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:22.798656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3f0a2a06-d488-4e2c-bd19-c38bbfaf4f6c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:22.799381 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3f0a2a06-d488-4e2c-bd19-c38bbfaf4f6c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:22.799826 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3f0a2a06-d488-4e2c-bd19-c38bbfaf4f6c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:22.800261 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3f0a2a06-d488-4e2c-bd19-c38bbfaf4f6c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:22.891327 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3f0a2a06-d488-4e2c-bd19-c38bbfaf4f6c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:22.902283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3f0a2a06-d488-4e2c-bd19-c38bbfaf4f6c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.115610 Aug 07 01:19:22.902902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 460/950] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:22 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:23.049990 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b47fb47d-3b49-4e0b-9813-20c51da3a9b2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:23.052403 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b47fb47d-3b49-4e0b-9813-20c51da3a9b2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:23.060097 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b47fb47d-3b49-4e0b-9813-20c51da3a9b2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:23.060704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b47fb47d-3b49-4e0b-9813-20c51da3a9b2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:23.061196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b47fb47d-3b49-4e0b-9813-20c51da3a9b2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:23.062012 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b47fb47d-3b49-4e0b-9813-20c51da3a9b2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:23.062545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b47fb47d-3b49-4e0b-9813-20c51da3a9b2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:23.063037 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b47fb47d-3b49-4e0b-9813-20c51da3a9b2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:23.099145 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b47fb47d-3b49-4e0b-9813-20c51da3a9b2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054202 Aug 07 01:19:23.099772 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 491/951] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:23 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:23.929129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d7dd5fe7-7239-4f07-bc2c-931f5477108a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:23.930310 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d7dd5fe7-7239-4f07-bc2c-931f5477108a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:23.943758 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7dd5fe7-7239-4f07-bc2c-931f5477108a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:23.944662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7dd5fe7-7239-4f07-bc2c-931f5477108a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:23.945503 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7dd5fe7-7239-4f07-bc2c-931f5477108a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:23.947319 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7dd5fe7-7239-4f07-bc2c-931f5477108a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:23.948154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7dd5fe7-7239-4f07-bc2c-931f5477108a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:23.949061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7dd5fe7-7239-4f07-bc2c-931f5477108a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:24.051837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d7dd5fe7-7239-4f07-bc2c-931f5477108a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:24.061023 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d7dd5fe7-7239-4f07-bc2c-931f5477108a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.136974 Aug 07 01:19:24.061782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 461/952] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:23 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:24.117247 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0487e08c-fb1e-4b7e-aef2-7b22cfd13c3d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:24.120541 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0487e08c-fb1e-4b7e-aef2-7b22cfd13c3d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:24.128326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0487e08c-fb1e-4b7e-aef2-7b22cfd13c3d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:24.128974 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0487e08c-fb1e-4b7e-aef2-7b22cfd13c3d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:24.129553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0487e08c-fb1e-4b7e-aef2-7b22cfd13c3d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:24.130444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0487e08c-fb1e-4b7e-aef2-7b22cfd13c3d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:24.130965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0487e08c-fb1e-4b7e-aef2-7b22cfd13c3d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:24.131442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0487e08c-fb1e-4b7e-aef2-7b22cfd13c3d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:24.180355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0487e08c-fb1e-4b7e-aef2-7b22cfd13c3d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067942 Aug 07 01:19:24.180918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 492/953] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:24 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:25.085410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f5a4460e-7993-4ee4-873c-3a3120222a22 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:25.086065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f5a4460e-7993-4ee4-873c-3a3120222a22 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:25.093656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5a4460e-7993-4ee4-873c-3a3120222a22 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:25.094202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5a4460e-7993-4ee4-873c-3a3120222a22 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:25.094691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5a4460e-7993-4ee4-873c-3a3120222a22 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:25.095519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5a4460e-7993-4ee4-873c-3a3120222a22 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:25.096071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5a4460e-7993-4ee4-873c-3a3120222a22 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:25.096564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f5a4460e-7993-4ee4-873c-3a3120222a22 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:25.170467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f5a4460e-7993-4ee4-873c-3a3120222a22 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:25.180492 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f5a4460e-7993-4ee4-873c-3a3120222a22 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.099095 Aug 07 01:19:25.181090 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 462/954] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:25 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:25.195139 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-32ebea8b-b013-45b0-9a0d-f2c64ebd16f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:25.199655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-32ebea8b-b013-45b0-9a0d-f2c64ebd16f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:25.212166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32ebea8b-b013-45b0-9a0d-f2c64ebd16f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:25.213733 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32ebea8b-b013-45b0-9a0d-f2c64ebd16f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:25.214788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32ebea8b-b013-45b0-9a0d-f2c64ebd16f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:25.216106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32ebea8b-b013-45b0-9a0d-f2c64ebd16f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:25.217244 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32ebea8b-b013-45b0-9a0d-f2c64ebd16f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:25.218098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32ebea8b-b013-45b0-9a0d-f2c64ebd16f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:25.270092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-32ebea8b-b013-45b0-9a0d-f2c64ebd16f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.079096 Aug 07 01:19:25.270937 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 493/955] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:25 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:26.226535 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-596f6a02-2b61-441d-be40-cb799ac135db tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:26.227568 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-596f6a02-2b61-441d-be40-cb799ac135db tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:26.235599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-596f6a02-2b61-441d-be40-cb799ac135db tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:26.236097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-596f6a02-2b61-441d-be40-cb799ac135db tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:26.236582 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-596f6a02-2b61-441d-be40-cb799ac135db tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:26.237349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-596f6a02-2b61-441d-be40-cb799ac135db tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:26.237904 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-596f6a02-2b61-441d-be40-cb799ac135db tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:26.238339 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-596f6a02-2b61-441d-be40-cb799ac135db tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:26.291428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ff986c9e-a643-4afa-8174-ac2454f299c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:26.295707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ff986c9e-a643-4afa-8174-ac2454f299c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:26.309274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ff986c9e-a643-4afa-8174-ac2454f299c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:26.310158 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ff986c9e-a643-4afa-8174-ac2454f299c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:26.311010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ff986c9e-a643-4afa-8174-ac2454f299c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:26.312423 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ff986c9e-a643-4afa-8174-ac2454f299c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:26.313319 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ff986c9e-a643-4afa-8174-ac2454f299c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:26.314124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ff986c9e-a643-4afa-8174-ac2454f299c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:26.339239 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-596f6a02-2b61-441d-be40-cb799ac135db tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:26.348780 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-596f6a02-2b61-441d-be40-cb799ac135db tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.126661 Aug 07 01:19:26.349728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 463/956] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:26 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:26.361585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ff986c9e-a643-4afa-8174-ac2454f299c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.078694 Aug 07 01:19:26.362633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 494/957] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:26 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:27.378225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-16efe61a-dcab-445d-90cd-15b5c6406369 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:27.380342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c27a52cd-10de-4948-9a23-28358844319d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:27.381055 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-16efe61a-dcab-445d-90cd-15b5c6406369 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:27.381218 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c27a52cd-10de-4948-9a23-28358844319d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:27.388929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16efe61a-dcab-445d-90cd-15b5c6406369 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:27.389230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c27a52cd-10de-4948-9a23-28358844319d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:27.389433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16efe61a-dcab-445d-90cd-15b5c6406369 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:27.389609 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c27a52cd-10de-4948-9a23-28358844319d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:27.389912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16efe61a-dcab-445d-90cd-15b5c6406369 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:27.390093 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c27a52cd-10de-4948-9a23-28358844319d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:27.390590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16efe61a-dcab-445d-90cd-15b5c6406369 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:27.390776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c27a52cd-10de-4948-9a23-28358844319d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:27.391045 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16efe61a-dcab-445d-90cd-15b5c6406369 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:27.391234 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c27a52cd-10de-4948-9a23-28358844319d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:27.391468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16efe61a-dcab-445d-90cd-15b5c6406369 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:27.391666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c27a52cd-10de-4948-9a23-28358844319d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:27.441743 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-16efe61a-dcab-445d-90cd-15b5c6406369 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066992 Aug 07 01:19:27.442843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 464/958] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:27 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:27.489120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c27a52cd-10de-4948-9a23-28358844319d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:27.497863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c27a52cd-10de-4948-9a23-28358844319d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.121724 Aug 07 01:19:27.498540 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 495/959] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:27 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:28.456037 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ef04d57a-db22-483c-be2b-9a76afcc8bb5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:28.458363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ef04d57a-db22-483c-be2b-9a76afcc8bb5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:28.465948 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef04d57a-db22-483c-be2b-9a76afcc8bb5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:28.466442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef04d57a-db22-483c-be2b-9a76afcc8bb5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:28.466952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef04d57a-db22-483c-be2b-9a76afcc8bb5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:28.467719 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef04d57a-db22-483c-be2b-9a76afcc8bb5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:28.468170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef04d57a-db22-483c-be2b-9a76afcc8bb5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:28.468667 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef04d57a-db22-483c-be2b-9a76afcc8bb5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:28.504261 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ef04d57a-db22-483c-be2b-9a76afcc8bb5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052157 Aug 07 01:19:28.504853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 465/960] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:28 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:28.520502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-49c51655-351b-45d6-91fa-78e49e5cabbe tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:28.521339 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-49c51655-351b-45d6-91fa-78e49e5cabbe tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:28.528704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49c51655-351b-45d6-91fa-78e49e5cabbe tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:28.529198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49c51655-351b-45d6-91fa-78e49e5cabbe tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:28.529632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49c51655-351b-45d6-91fa-78e49e5cabbe tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:28.530380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49c51655-351b-45d6-91fa-78e49e5cabbe tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:28.530836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49c51655-351b-45d6-91fa-78e49e5cabbe tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:28.531266 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49c51655-351b-45d6-91fa-78e49e5cabbe tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:28.627159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-49c51655-351b-45d6-91fa-78e49e5cabbe tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:28.636988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-49c51655-351b-45d6-91fa-78e49e5cabbe tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.120429 Aug 07 01:19:28.637604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 496/961] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:28 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:29.516848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6bc77812-4cc4-4c41-9809-6ff2f896aff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:29.519415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6bc77812-4cc4-4c41-9809-6ff2f896aff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:29.528214 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6bc77812-4cc4-4c41-9809-6ff2f896aff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:29.528472 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6bc77812-4cc4-4c41-9809-6ff2f896aff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:29.529082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6bc77812-4cc4-4c41-9809-6ff2f896aff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:29.529805 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6bc77812-4cc4-4c41-9809-6ff2f896aff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:29.530594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6bc77812-4cc4-4c41-9809-6ff2f896aff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:29.531063 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6bc77812-4cc4-4c41-9809-6ff2f896aff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:29.565956 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6bc77812-4cc4-4c41-9809-6ff2f896aff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053033 Aug 07 01:19:29.566543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 466/962] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:29 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:29.659451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f43a784d-3e42-448b-9e42-5d6e763ac3b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:29.660154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f43a784d-3e42-448b-9e42-5d6e763ac3b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:29.667531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f43a784d-3e42-448b-9e42-5d6e763ac3b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:29.668197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f43a784d-3e42-448b-9e42-5d6e763ac3b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:29.668820 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f43a784d-3e42-448b-9e42-5d6e763ac3b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:29.669569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f43a784d-3e42-448b-9e42-5d6e763ac3b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:29.670085 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f43a784d-3e42-448b-9e42-5d6e763ac3b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:29.671727 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f43a784d-3e42-448b-9e42-5d6e763ac3b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:29.676769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:19:29.750529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f43a784d-3e42-448b-9e42-5d6e763ac3b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:29.766190 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f43a784d-3e42-448b-9e42-5d6e763ac3b6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.110443 Aug 07 01:19:29.767100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 497/963] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:29 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:30.582262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-75d8b4ac-956c-484b-99dc-f7c3e841746e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:30.584613 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-75d8b4ac-956c-484b-99dc-f7c3e841746e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:30.591837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-75d8b4ac-956c-484b-99dc-f7c3e841746e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:30.592488 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-75d8b4ac-956c-484b-99dc-f7c3e841746e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:30.592982 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-75d8b4ac-956c-484b-99dc-f7c3e841746e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:30.593966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-75d8b4ac-956c-484b-99dc-f7c3e841746e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:30.594652 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-75d8b4ac-956c-484b-99dc-f7c3e841746e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:30.595275 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-75d8b4ac-956c-484b-99dc-f7c3e841746e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:30.630404 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-75d8b4ac-956c-484b-99dc-f7c3e841746e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052146 Aug 07 01:19:30.630997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 467/964] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:30 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:30.795088 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7bc26e24-965c-4ad5-af27-c2e09412e6d6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:30.795738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7bc26e24-965c-4ad5-af27-c2e09412e6d6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:30.798449 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=56 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:19:30.804849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bc26e24-965c-4ad5-af27-c2e09412e6d6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:30.805359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bc26e24-965c-4ad5-af27-c2e09412e6d6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:30.806319 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bc26e24-965c-4ad5-af27-c2e09412e6d6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:30.807091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bc26e24-965c-4ad5-af27-c2e09412e6d6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:30.807549 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bc26e24-965c-4ad5-af27-c2e09412e6d6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:30.807971 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bc26e24-965c-4ad5-af27-c2e09412e6d6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:30.880038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7bc26e24-965c-4ad5-af27-c2e09412e6d6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:30.888475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7bc26e24-965c-4ad5-af27-c2e09412e6d6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097671 Aug 07 01:19:30.889069 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 498/965] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:30 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:31.643873 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-63ad1eb1-a46c-4a05-8d0c-9717cd22e185 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:31.646515 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-63ad1eb1-a46c-4a05-8d0c-9717cd22e185 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:31.654011 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-63ad1eb1-a46c-4a05-8d0c-9717cd22e185 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:31.654498 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-63ad1eb1-a46c-4a05-8d0c-9717cd22e185 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:31.654933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-63ad1eb1-a46c-4a05-8d0c-9717cd22e185 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:31.655658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-63ad1eb1-a46c-4a05-8d0c-9717cd22e185 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:31.656172 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-63ad1eb1-a46c-4a05-8d0c-9717cd22e185 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:31.656662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-63ad1eb1-a46c-4a05-8d0c-9717cd22e185 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:31.693055 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-63ad1eb1-a46c-4a05-8d0c-9717cd22e185 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053521 Aug 07 01:19:31.693645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 468/966] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:31 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:31.910244 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-36232b21-8a1d-4447-b4c7-3bd87be48895 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:31.910912 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-36232b21-8a1d-4447-b4c7-3bd87be48895 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:31.918622 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36232b21-8a1d-4447-b4c7-3bd87be48895 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:31.919112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36232b21-8a1d-4447-b4c7-3bd87be48895 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:31.919550 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36232b21-8a1d-4447-b4c7-3bd87be48895 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:31.920292 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36232b21-8a1d-4447-b4c7-3bd87be48895 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:31.920748 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36232b21-8a1d-4447-b4c7-3bd87be48895 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:31.921173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36232b21-8a1d-4447-b4c7-3bd87be48895 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:32.003235 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-36232b21-8a1d-4447-b4c7-3bd87be48895 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:32.011747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-36232b21-8a1d-4447-b4c7-3bd87be48895 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.105787 Aug 07 01:19:32.012475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 499/967] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:31 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:32.706880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-dc41ac23-37a8-424f-a0cd-b7689343ff7e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:32.709097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-dc41ac23-37a8-424f-a0cd-b7689343ff7e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:32.711643 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=54 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:19:32.718005 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc41ac23-37a8-424f-a0cd-b7689343ff7e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:32.718494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc41ac23-37a8-424f-a0cd-b7689343ff7e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:32.718944 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc41ac23-37a8-424f-a0cd-b7689343ff7e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:32.719679 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc41ac23-37a8-424f-a0cd-b7689343ff7e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:32.720132 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc41ac23-37a8-424f-a0cd-b7689343ff7e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:32.720594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc41ac23-37a8-424f-a0cd-b7689343ff7e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:32.756649 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-dc41ac23-37a8-424f-a0cd-b7689343ff7e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054002 Aug 07 01:19:32.757231 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 469/968] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:32 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:33.035066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-204dc997-d293-44b6-93fc-9fbf3502ac4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:33.035716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-204dc997-d293-44b6-93fc-9fbf3502ac4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:33.043350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-204dc997-d293-44b6-93fc-9fbf3502ac4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:33.043900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-204dc997-d293-44b6-93fc-9fbf3502ac4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:33.044463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-204dc997-d293-44b6-93fc-9fbf3502ac4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:33.045248 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-204dc997-d293-44b6-93fc-9fbf3502ac4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:33.045718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-204dc997-d293-44b6-93fc-9fbf3502ac4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:33.046149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-204dc997-d293-44b6-93fc-9fbf3502ac4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:33.135924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-204dc997-d293-44b6-93fc-9fbf3502ac4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:33.149182 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-204dc997-d293-44b6-93fc-9fbf3502ac4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.118269 Aug 07 01:19:33.149924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 500/969] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:33 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:33.778051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cf6c4551-b7ac-4f26-b969-d169a1ad0d08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:33.781548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-cf6c4551-b7ac-4f26-b969-d169a1ad0d08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:33.794901 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf6c4551-b7ac-4f26-b969-d169a1ad0d08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:33.795776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf6c4551-b7ac-4f26-b969-d169a1ad0d08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:33.796572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf6c4551-b7ac-4f26-b969-d169a1ad0d08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:33.797725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf6c4551-b7ac-4f26-b969-d169a1ad0d08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:33.798451 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf6c4551-b7ac-4f26-b969-d169a1ad0d08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:33.799180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf6c4551-b7ac-4f26-b969-d169a1ad0d08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:33.846249 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-cf6c4551-b7ac-4f26-b969-d169a1ad0d08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.076817 Aug 07 01:19:33.847231 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 470/970] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:33 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:34.178489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d46cf2ff-413e-4375-87f4-2dc3fefd521f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:34.179092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d46cf2ff-413e-4375-87f4-2dc3fefd521f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:34.186676 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d46cf2ff-413e-4375-87f4-2dc3fefd521f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:34.187170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d46cf2ff-413e-4375-87f4-2dc3fefd521f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:34.187610 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d46cf2ff-413e-4375-87f4-2dc3fefd521f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:34.188465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d46cf2ff-413e-4375-87f4-2dc3fefd521f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:34.188929 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d46cf2ff-413e-4375-87f4-2dc3fefd521f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:34.189353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d46cf2ff-413e-4375-87f4-2dc3fefd521f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:34.266157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d46cf2ff-413e-4375-87f4-2dc3fefd521f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:34.280520 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d46cf2ff-413e-4375-87f4-2dc3fefd521f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.106454 Aug 07 01:19:34.281437 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 501/971] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:34 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:34.862526 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-afc5f310-0a41-4d32-98bd-bf8d6d67af83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:34.865025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-afc5f310-0a41-4d32-98bd-bf8d6d67af83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:34.873175 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-afc5f310-0a41-4d32-98bd-bf8d6d67af83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:34.873762 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-afc5f310-0a41-4d32-98bd-bf8d6d67af83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:34.874243 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-afc5f310-0a41-4d32-98bd-bf8d6d67af83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:34.875043 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-afc5f310-0a41-4d32-98bd-bf8d6d67af83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:34.875531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-afc5f310-0a41-4d32-98bd-bf8d6d67af83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:34.875952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-afc5f310-0a41-4d32-98bd-bf8d6d67af83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:34.915890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-afc5f310-0a41-4d32-98bd-bf8d6d67af83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057105 Aug 07 01:19:34.916576 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 471/972] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:34 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:35.318740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-24d293f7-7724-4de1-8595-e510defe0d2b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:35.319407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-24d293f7-7724-4de1-8595-e510defe0d2b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:35.326930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-24d293f7-7724-4de1-8595-e510defe0d2b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:35.327420 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-24d293f7-7724-4de1-8595-e510defe0d2b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:35.327860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-24d293f7-7724-4de1-8595-e510defe0d2b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:35.328610 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-24d293f7-7724-4de1-8595-e510defe0d2b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:35.329060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-24d293f7-7724-4de1-8595-e510defe0d2b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:35.329484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-24d293f7-7724-4de1-8595-e510defe0d2b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:35.407861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-24d293f7-7724-4de1-8595-e510defe0d2b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:35.422325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-24d293f7-7724-4de1-8595-e510defe0d2b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.107157 Aug 07 01:19:35.423413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 502/973] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:35 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:35.935773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-82c8bf2f-e1a1-4381-ac81-c040ca876e5e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:35.938568 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-82c8bf2f-e1a1-4381-ac81-c040ca876e5e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:35.945620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-82c8bf2f-e1a1-4381-ac81-c040ca876e5e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:35.946174 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-82c8bf2f-e1a1-4381-ac81-c040ca876e5e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:35.947140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-82c8bf2f-e1a1-4381-ac81-c040ca876e5e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:35.947865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-82c8bf2f-e1a1-4381-ac81-c040ca876e5e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:35.948363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-82c8bf2f-e1a1-4381-ac81-c040ca876e5e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:35.948816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-82c8bf2f-e1a1-4381-ac81-c040ca876e5e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:35.967751 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:19:35.983046 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-82c8bf2f-e1a1-4381-ac81-c040ca876e5e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052217 Aug 07 01:19:35.983491 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 472/974] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:35 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:36.448088 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ee07e98a-9eb9-46d8-b3ad-d59e45764e44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:36.448786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ee07e98a-9eb9-46d8-b3ad-d59e45764e44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:36.457249 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee07e98a-9eb9-46d8-b3ad-d59e45764e44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:36.457920 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee07e98a-9eb9-46d8-b3ad-d59e45764e44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:36.458248 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee07e98a-9eb9-46d8-b3ad-d59e45764e44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:36.458993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee07e98a-9eb9-46d8-b3ad-d59e45764e44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:36.459465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee07e98a-9eb9-46d8-b3ad-d59e45764e44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:36.459898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee07e98a-9eb9-46d8-b3ad-d59e45764e44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:36.549708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ee07e98a-9eb9-46d8-b3ad-d59e45764e44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:36.559714 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ee07e98a-9eb9-46d8-b3ad-d59e45764e44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.116197 Aug 07 01:19:36.560463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 503/975] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:36 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:36.998686 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c65a9d15-6085-4db7-a94e-73784d4adc36 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:37.001134 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c65a9d15-6085-4db7-a94e-73784d4adc36 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:37.011065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c65a9d15-6085-4db7-a94e-73784d4adc36 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:37.011636 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c65a9d15-6085-4db7-a94e-73784d4adc36 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:37.012117 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c65a9d15-6085-4db7-a94e-73784d4adc36 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:37.012930 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c65a9d15-6085-4db7-a94e-73784d4adc36 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:37.013386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c65a9d15-6085-4db7-a94e-73784d4adc36 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:37.013812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c65a9d15-6085-4db7-a94e-73784d4adc36 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:37.048397 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c65a9d15-6085-4db7-a94e-73784d4adc36 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056470 Aug 07 01:19:37.049047 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 473/976] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:36 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:37.581674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5ec7d1cc-1fe1-4560-b8f2-7a37a31f39ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:37.582569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5ec7d1cc-1fe1-4560-b8f2-7a37a31f39ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:37.590855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5ec7d1cc-1fe1-4560-b8f2-7a37a31f39ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:37.591336 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5ec7d1cc-1fe1-4560-b8f2-7a37a31f39ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:37.591809 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5ec7d1cc-1fe1-4560-b8f2-7a37a31f39ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:37.592540 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5ec7d1cc-1fe1-4560-b8f2-7a37a31f39ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:37.592996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5ec7d1cc-1fe1-4560-b8f2-7a37a31f39ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:37.593428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5ec7d1cc-1fe1-4560-b8f2-7a37a31f39ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:37.666143 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5ec7d1cc-1fe1-4560-b8f2-7a37a31f39ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:37.674611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5ec7d1cc-1fe1-4560-b8f2-7a37a31f39ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097348 Aug 07 01:19:37.675479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 504/977] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:37 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:38.062118 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c47f14ae-8467-4a3e-bf46-49a799ca4323 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:38.064375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c47f14ae-8467-4a3e-bf46-49a799ca4323 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:38.072022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c47f14ae-8467-4a3e-bf46-49a799ca4323 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:38.072634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c47f14ae-8467-4a3e-bf46-49a799ca4323 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:38.073073 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c47f14ae-8467-4a3e-bf46-49a799ca4323 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:38.073811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c47f14ae-8467-4a3e-bf46-49a799ca4323 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:38.074271 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c47f14ae-8467-4a3e-bf46-49a799ca4323 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:38.074696 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c47f14ae-8467-4a3e-bf46-49a799ca4323 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:38.111591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c47f14ae-8467-4a3e-bf46-49a799ca4323 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053449 Aug 07 01:19:38.112532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 474/978] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:38 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:38.696968 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5fe85250-157f-408f-9af2-523052dfff08 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:38.697913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5fe85250-157f-408f-9af2-523052dfff08 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:38.706034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fe85250-157f-408f-9af2-523052dfff08 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:38.706528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fe85250-157f-408f-9af2-523052dfff08 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:38.706971 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fe85250-157f-408f-9af2-523052dfff08 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:38.707705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fe85250-157f-408f-9af2-523052dfff08 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:38.708166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fe85250-157f-408f-9af2-523052dfff08 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:38.708628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fe85250-157f-408f-9af2-523052dfff08 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:38.815061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5fe85250-157f-408f-9af2-523052dfff08 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:38.824226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5fe85250-157f-408f-9af2-523052dfff08 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.131644 Aug 07 01:19:38.824858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 505/979] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:38 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:39.126055 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1a62c148-6ab9-4cf8-90f2-9a2f13b188ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:39.128939 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1a62c148-6ab9-4cf8-90f2-9a2f13b188ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:39.137159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a62c148-6ab9-4cf8-90f2-9a2f13b188ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:39.137652 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a62c148-6ab9-4cf8-90f2-9a2f13b188ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:39.138090 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a62c148-6ab9-4cf8-90f2-9a2f13b188ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:39.138812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a62c148-6ab9-4cf8-90f2-9a2f13b188ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:39.139267 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a62c148-6ab9-4cf8-90f2-9a2f13b188ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:39.139695 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a62c148-6ab9-4cf8-90f2-9a2f13b188ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:39.184575 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1a62c148-6ab9-4cf8-90f2-9a2f13b188ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062854 Aug 07 01:19:39.185637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 475/980] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:39 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:39.860213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c7c0f462-abf7-46ba-93d0-f5c1ea799f45 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:39.861095 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c7c0f462-abf7-46ba-93d0-f5c1ea799f45 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:39.868292 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c7c0f462-abf7-46ba-93d0-f5c1ea799f45 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:39.868892 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c7c0f462-abf7-46ba-93d0-f5c1ea799f45 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:39.869323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c7c0f462-abf7-46ba-93d0-f5c1ea799f45 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:39.870084 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c7c0f462-abf7-46ba-93d0-f5c1ea799f45 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:39.870584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c7c0f462-abf7-46ba-93d0-f5c1ea799f45 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:39.871058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c7c0f462-abf7-46ba-93d0-f5c1ea799f45 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:40.104201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c7c0f462-abf7-46ba-93d0-f5c1ea799f45 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:40.113044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c7c0f462-abf7-46ba-93d0-f5c1ea799f45 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.256721 Aug 07 01:19:40.113789 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 506/981] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:39 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:40.199756 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-085458e1-9aad-49c8-b800-5aa28c8ef5f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:40.202409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-085458e1-9aad-49c8-b800-5aa28c8ef5f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:40.210989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-085458e1-9aad-49c8-b800-5aa28c8ef5f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:40.212173 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-085458e1-9aad-49c8-b800-5aa28c8ef5f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:40.212786 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-085458e1-9aad-49c8-b800-5aa28c8ef5f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:40.213764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-085458e1-9aad-49c8-b800-5aa28c8ef5f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:40.214433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-085458e1-9aad-49c8-b800-5aa28c8ef5f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:40.215031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-085458e1-9aad-49c8-b800-5aa28c8ef5f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:40.248080 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-085458e1-9aad-49c8-b800-5aa28c8ef5f7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052500 Aug 07 01:19:40.248903 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 476/982] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:40 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:41.136309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f871987e-ce13-4e65-b646-6178e151a596 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:41.137144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f871987e-ce13-4e65-b646-6178e151a596 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:41.145300 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f871987e-ce13-4e65-b646-6178e151a596 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:41.145799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f871987e-ce13-4e65-b646-6178e151a596 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:41.146240 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f871987e-ce13-4e65-b646-6178e151a596 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:41.147034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f871987e-ce13-4e65-b646-6178e151a596 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:41.147858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f871987e-ce13-4e65-b646-6178e151a596 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:41.148414 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f871987e-ce13-4e65-b646-6178e151a596 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:41.221459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f871987e-ce13-4e65-b646-6178e151a596 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:41.230465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f871987e-ce13-4e65-b646-6178e151a596 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.098820 Aug 07 01:19:41.231081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 507/983] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:41 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:41.261006 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-349fe7b7-30a4-4f9f-b291-8bd291f4038d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:41.263422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-349fe7b7-30a4-4f9f-b291-8bd291f4038d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:41.271243 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-349fe7b7-30a4-4f9f-b291-8bd291f4038d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:41.271737 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-349fe7b7-30a4-4f9f-b291-8bd291f4038d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:41.272174 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-349fe7b7-30a4-4f9f-b291-8bd291f4038d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:41.273013 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-349fe7b7-30a4-4f9f-b291-8bd291f4038d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:41.273481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-349fe7b7-30a4-4f9f-b291-8bd291f4038d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:41.274433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-349fe7b7-30a4-4f9f-b291-8bd291f4038d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:41.310888 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-349fe7b7-30a4-4f9f-b291-8bd291f4038d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054251 Aug 07 01:19:41.311489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 477/984] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:41 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:42.262557 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b47e4dbd-0f1c-4461-b08e-70d30e378bd2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:42.263236 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b47e4dbd-0f1c-4461-b08e-70d30e378bd2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:42.270817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b47e4dbd-0f1c-4461-b08e-70d30e378bd2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:42.271380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b47e4dbd-0f1c-4461-b08e-70d30e378bd2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:42.271810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b47e4dbd-0f1c-4461-b08e-70d30e378bd2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:42.272609 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b47e4dbd-0f1c-4461-b08e-70d30e378bd2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:42.273123 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b47e4dbd-0f1c-4461-b08e-70d30e378bd2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:42.273591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b47e4dbd-0f1c-4461-b08e-70d30e378bd2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:42.331798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-af706c26-0277-48e6-827f-72bfdc199447 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:42.335572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-af706c26-0277-48e6-827f-72bfdc199447 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:42.347252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b47e4dbd-0f1c-4461-b08e-70d30e378bd2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:42.348174 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af706c26-0277-48e6-827f-72bfdc199447 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:42.349084 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af706c26-0277-48e6-827f-72bfdc199447 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:42.349877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af706c26-0277-48e6-827f-72bfdc199447 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:42.351185 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af706c26-0277-48e6-827f-72bfdc199447 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:42.352110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af706c26-0277-48e6-827f-72bfdc199447 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:42.352951 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af706c26-0277-48e6-827f-72bfdc199447 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:42.355638 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b47e4dbd-0f1c-4461-b08e-70d30e378bd2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097368 Aug 07 01:19:42.360294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 508/985] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:42 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:42.412912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-af706c26-0277-48e6-827f-72bfdc199447 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.088712 Aug 07 01:19:42.414418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 478/986] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:42 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:43.377706 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-483ec71a-cfa2-4f07-8bc2-d58caae1e5f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:43.378334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-483ec71a-cfa2-4f07-8bc2-d58caae1e5f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:43.386251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-483ec71a-cfa2-4f07-8bc2-d58caae1e5f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:43.386744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-483ec71a-cfa2-4f07-8bc2-d58caae1e5f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:43.387216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-483ec71a-cfa2-4f07-8bc2-d58caae1e5f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:43.388020 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-483ec71a-cfa2-4f07-8bc2-d58caae1e5f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:43.388531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-483ec71a-cfa2-4f07-8bc2-d58caae1e5f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:43.388977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-483ec71a-cfa2-4f07-8bc2-d58caae1e5f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:43.430991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c97bad0f-160f-440a-a3d3-515b6bfcfc52 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:43.433419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c97bad0f-160f-440a-a3d3-515b6bfcfc52 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:43.440696 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c97bad0f-160f-440a-a3d3-515b6bfcfc52 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:43.441280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c97bad0f-160f-440a-a3d3-515b6bfcfc52 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:43.441834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c97bad0f-160f-440a-a3d3-515b6bfcfc52 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:43.442647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c97bad0f-160f-440a-a3d3-515b6bfcfc52 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:43.443180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c97bad0f-160f-440a-a3d3-515b6bfcfc52 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:43.443689 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c97bad0f-160f-440a-a3d3-515b6bfcfc52 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:43.479157 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c97bad0f-160f-440a-a3d3-515b6bfcfc52 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053311 Aug 07 01:19:43.479802 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 479/987] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:43 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:43.482274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-483ec71a-cfa2-4f07-8bc2-d58caae1e5f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:43.497872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-483ec71a-cfa2-4f07-8bc2-d58caae1e5f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.124155 Aug 07 01:19:43.498880 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 509/988] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:43 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:44.495093 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-34c9bf12-cf13-4380-8b24-4ff4c2d580cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:44.497532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-34c9bf12-cf13-4380-8b24-4ff4c2d580cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:44.505441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34c9bf12-cf13-4380-8b24-4ff4c2d580cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:44.506016 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34c9bf12-cf13-4380-8b24-4ff4c2d580cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:44.506543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34c9bf12-cf13-4380-8b24-4ff4c2d580cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:44.507349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34c9bf12-cf13-4380-8b24-4ff4c2d580cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:44.507890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34c9bf12-cf13-4380-8b24-4ff4c2d580cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:44.508901 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34c9bf12-cf13-4380-8b24-4ff4c2d580cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:44.533562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-870bc509-a266-49f5-b318-17593244eda2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:44.534356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-870bc509-a266-49f5-b318-17593244eda2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:44.550982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-870bc509-a266-49f5-b318-17593244eda2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:44.551806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-870bc509-a266-49f5-b318-17593244eda2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:44.552543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-870bc509-a266-49f5-b318-17593244eda2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:44.553724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-870bc509-a266-49f5-b318-17593244eda2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:44.554478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-870bc509-a266-49f5-b318-17593244eda2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:44.555263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-870bc509-a266-49f5-b318-17593244eda2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:44.571284 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-34c9bf12-cf13-4380-8b24-4ff4c2d580cd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.079563 Aug 07 01:19:44.572709 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 480/989] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:44 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:44.647380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-870bc509-a266-49f5-b318-17593244eda2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:44.662286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-870bc509-a266-49f5-b318-17593244eda2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.132682 Aug 07 01:19:44.663302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 510/990] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:44 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:45.591268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-599157e3-2f93-4f38-95cc-67ec2a29a896 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:45.595178 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-599157e3-2f93-4f38-95cc-67ec2a29a896 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:45.607640 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-599157e3-2f93-4f38-95cc-67ec2a29a896 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:45.608567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-599157e3-2f93-4f38-95cc-67ec2a29a896 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:45.609363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-599157e3-2f93-4f38-95cc-67ec2a29a896 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:45.610688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-599157e3-2f93-4f38-95cc-67ec2a29a896 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:45.611520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-599157e3-2f93-4f38-95cc-67ec2a29a896 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:45.612338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-599157e3-2f93-4f38-95cc-67ec2a29a896 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:45.670377 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-599157e3-2f93-4f38-95cc-67ec2a29a896 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.088453 Aug 07 01:19:45.671454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 481/991] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:45 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:45.693751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-420f47c7-effd-4b5d-a595-93db5547c9dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:45.694416 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-420f47c7-effd-4b5d-a595-93db5547c9dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:45.704026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-420f47c7-effd-4b5d-a595-93db5547c9dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:45.704539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-420f47c7-effd-4b5d-a595-93db5547c9dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:45.704978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-420f47c7-effd-4b5d-a595-93db5547c9dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:45.705857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-420f47c7-effd-4b5d-a595-93db5547c9dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:45.706168 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-420f47c7-effd-4b5d-a595-93db5547c9dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:45.706590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-420f47c7-effd-4b5d-a595-93db5547c9dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:45.775227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-420f47c7-effd-4b5d-a595-93db5547c9dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:45.784013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-420f47c7-effd-4b5d-a595-93db5547c9dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.094311 Aug 07 01:19:45.784546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 511/992] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:45 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:46.687238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2a8a844a-9c30-4112-8603-c2c8612cb06a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:46.689605 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2a8a844a-9c30-4112-8603-c2c8612cb06a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:46.696540 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a8a844a-9c30-4112-8603-c2c8612cb06a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:46.697031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a8a844a-9c30-4112-8603-c2c8612cb06a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:46.697462 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a8a844a-9c30-4112-8603-c2c8612cb06a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:46.698185 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a8a844a-9c30-4112-8603-c2c8612cb06a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:46.698676 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a8a844a-9c30-4112-8603-c2c8612cb06a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:46.699286 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a8a844a-9c30-4112-8603-c2c8612cb06a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:46.734864 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2a8a844a-9c30-4112-8603-c2c8612cb06a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051929 Aug 07 01:19:46.735462 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 482/993] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:46 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:46.807037 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-31d51e3c-46e6-43fd-9a78-ff0b77f7f975 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:46.808473 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-31d51e3c-46e6-43fd-9a78-ff0b77f7f975 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:46.815893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31d51e3c-46e6-43fd-9a78-ff0b77f7f975 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:46.816451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31d51e3c-46e6-43fd-9a78-ff0b77f7f975 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:46.816919 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31d51e3c-46e6-43fd-9a78-ff0b77f7f975 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:46.817646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31d51e3c-46e6-43fd-9a78-ff0b77f7f975 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:46.818106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31d51e3c-46e6-43fd-9a78-ff0b77f7f975 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:46.818535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31d51e3c-46e6-43fd-9a78-ff0b77f7f975 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:46.894320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-31d51e3c-46e6-43fd-9a78-ff0b77f7f975 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:46.904059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-31d51e3c-46e6-43fd-9a78-ff0b77f7f975 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.101700 Aug 07 01:19:46.904716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 512/994] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:46 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:47.751239 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e3f8b855-2452-4115-bd31-3f09d0c67cca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:47.754305 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e3f8b855-2452-4115-bd31-3f09d0c67cca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:47.761757 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3f8b855-2452-4115-bd31-3f09d0c67cca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:47.762168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3f8b855-2452-4115-bd31-3f09d0c67cca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:47.762605 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3f8b855-2452-4115-bd31-3f09d0c67cca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:47.763333 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3f8b855-2452-4115-bd31-3f09d0c67cca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:47.763783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3f8b855-2452-4115-bd31-3f09d0c67cca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:47.764206 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e3f8b855-2452-4115-bd31-3f09d0c67cca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:47.798754 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e3f8b855-2452-4115-bd31-3f09d0c67cca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051608 Aug 07 01:19:47.799444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 483/995] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:47 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:47.925452 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-49cf7b0f-619c-4ee3-892f-58b86b2fc309 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:47.926134 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-49cf7b0f-619c-4ee3-892f-58b86b2fc309 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:47.934203 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49cf7b0f-619c-4ee3-892f-58b86b2fc309 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:47.934774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49cf7b0f-619c-4ee3-892f-58b86b2fc309 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:47.935286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49cf7b0f-619c-4ee3-892f-58b86b2fc309 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:47.936029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49cf7b0f-619c-4ee3-892f-58b86b2fc309 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:47.936532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49cf7b0f-619c-4ee3-892f-58b86b2fc309 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:47.936963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-49cf7b0f-619c-4ee3-892f-58b86b2fc309 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:48.008640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-49cf7b0f-619c-4ee3-892f-58b86b2fc309 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:48.012267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:19:48.017462 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-49cf7b0f-619c-4ee3-892f-58b86b2fc309 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.095993 Aug 07 01:19:48.018013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 513/996] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:47 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:48.813806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1ea5a356-9bba-421e-88db-2f2fed3f199f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:48.816019 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1ea5a356-9bba-421e-88db-2f2fed3f199f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:48.823695 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ea5a356-9bba-421e-88db-2f2fed3f199f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:48.824182 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ea5a356-9bba-421e-88db-2f2fed3f199f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:48.824645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ea5a356-9bba-421e-88db-2f2fed3f199f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:48.825361 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ea5a356-9bba-421e-88db-2f2fed3f199f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:48.826069 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ea5a356-9bba-421e-88db-2f2fed3f199f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:48.826494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ea5a356-9bba-421e-88db-2f2fed3f199f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:48.883775 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1ea5a356-9bba-421e-88db-2f2fed3f199f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073586 Aug 07 01:19:48.884887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 484/997] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:48 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:49.039458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3dbfdd3b-4fb6-4a87-ad5b-bc7b04a40baf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:49.040211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3dbfdd3b-4fb6-4a87-ad5b-bc7b04a40baf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:49.047704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3dbfdd3b-4fb6-4a87-ad5b-bc7b04a40baf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:49.048299 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3dbfdd3b-4fb6-4a87-ad5b-bc7b04a40baf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:49.048852 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3dbfdd3b-4fb6-4a87-ad5b-bc7b04a40baf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:49.049578 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3dbfdd3b-4fb6-4a87-ad5b-bc7b04a40baf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:49.050039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3dbfdd3b-4fb6-4a87-ad5b-bc7b04a40baf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:49.050463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3dbfdd3b-4fb6-4a87-ad5b-bc7b04a40baf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:49.133517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3dbfdd3b-4fb6-4a87-ad5b-bc7b04a40baf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:49.143058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3dbfdd3b-4fb6-4a87-ad5b-bc7b04a40baf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.107698 Aug 07 01:19:49.143804 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 514/998] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:49 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:49.900430 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b5d18668-dab1-4579-9c53-d5e169248eab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:49.902704 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b5d18668-dab1-4579-9c53-d5e169248eab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:49.910208 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b5d18668-dab1-4579-9c53-d5e169248eab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:49.910748 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b5d18668-dab1-4579-9c53-d5e169248eab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:49.911251 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b5d18668-dab1-4579-9c53-d5e169248eab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:49.912032 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b5d18668-dab1-4579-9c53-d5e169248eab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:49.912673 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b5d18668-dab1-4579-9c53-d5e169248eab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:49.913194 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b5d18668-dab1-4579-9c53-d5e169248eab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:49.952398 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b5d18668-dab1-4579-9c53-d5e169248eab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057079 Aug 07 01:19:49.953003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 485/999] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:49 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:50.168893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dc7abd60-137d-4069-81c8-6333802cbbb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:50.169878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dc7abd60-137d-4069-81c8-6333802cbbb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:50.177308 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc7abd60-137d-4069-81c8-6333802cbbb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:50.177851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc7abd60-137d-4069-81c8-6333802cbbb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:50.178298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc7abd60-137d-4069-81c8-6333802cbbb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:50.179021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc7abd60-137d-4069-81c8-6333802cbbb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:50.179504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc7abd60-137d-4069-81c8-6333802cbbb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:50.179935 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc7abd60-137d-4069-81c8-6333802cbbb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:50.264537 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-dc7abd60-137d-4069-81c8-6333802cbbb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:50.279400 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dc7abd60-137d-4069-81c8-6333802cbbb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.116617 Aug 07 01:19:50.280506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 515/1000] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:50 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:50.967637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5ccc31a6-080e-472c-9825-398baaccda0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:50.969881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5ccc31a6-080e-472c-9825-398baaccda0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:50.976712 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5ccc31a6-080e-472c-9825-398baaccda0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:50.977204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5ccc31a6-080e-472c-9825-398baaccda0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:50.977647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5ccc31a6-080e-472c-9825-398baaccda0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:50.978417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5ccc31a6-080e-472c-9825-398baaccda0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:50.978911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5ccc31a6-080e-472c-9825-398baaccda0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:50.979361 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5ccc31a6-080e-472c-9825-398baaccda0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:51.013635 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5ccc31a6-080e-472c-9825-398baaccda0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050059 Aug 07 01:19:51.014228 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 486/1001] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:50 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:51.303407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1a19698c-d608-4391-9cd3-04ae20a60e2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:51.304185 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1a19698c-d608-4391-9cd3-04ae20a60e2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:51.312260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a19698c-d608-4391-9cd3-04ae20a60e2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:51.312765 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a19698c-d608-4391-9cd3-04ae20a60e2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:51.313216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a19698c-d608-4391-9cd3-04ae20a60e2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:51.313951 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a19698c-d608-4391-9cd3-04ae20a60e2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:51.314409 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a19698c-d608-4391-9cd3-04ae20a60e2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:51.314840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a19698c-d608-4391-9cd3-04ae20a60e2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:51.396794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1a19698c-d608-4391-9cd3-04ae20a60e2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:51.410909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1a19698c-d608-4391-9cd3-04ae20a60e2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.111158 Aug 07 01:19:51.411951 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 516/1002] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:51 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:52.029332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-554d6ee6-5880-4951-9c6d-91ccb5fefc21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:52.031748 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-554d6ee6-5880-4951-9c6d-91ccb5fefc21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:52.039591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-554d6ee6-5880-4951-9c6d-91ccb5fefc21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:52.040173 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-554d6ee6-5880-4951-9c6d-91ccb5fefc21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:52.040696 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-554d6ee6-5880-4951-9c6d-91ccb5fefc21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:52.041572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-554d6ee6-5880-4951-9c6d-91ccb5fefc21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:52.042103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-554d6ee6-5880-4951-9c6d-91ccb5fefc21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:52.042539 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-554d6ee6-5880-4951-9c6d-91ccb5fefc21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:52.079411 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-554d6ee6-5880-4951-9c6d-91ccb5fefc21 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054928 Aug 07 01:19:52.080024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 487/1003] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:52 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:52.436030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9129bf19-c73e-47f1-ad30-9fd2ab38f037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:52.437243 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9129bf19-c73e-47f1-ad30-9fd2ab38f037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:52.449571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9129bf19-c73e-47f1-ad30-9fd2ab38f037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:52.450212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9129bf19-c73e-47f1-ad30-9fd2ab38f037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:52.451121 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9129bf19-c73e-47f1-ad30-9fd2ab38f037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:52.452041 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9129bf19-c73e-47f1-ad30-9fd2ab38f037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:52.452701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9129bf19-c73e-47f1-ad30-9fd2ab38f037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:52.453356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9129bf19-c73e-47f1-ad30-9fd2ab38f037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:52.547771 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9129bf19-c73e-47f1-ad30-9fd2ab38f037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:52.561104 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9129bf19-c73e-47f1-ad30-9fd2ab38f037 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.131384 Aug 07 01:19:52.562108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 517/1004] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:52 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:53.097657 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4b30411b-40c6-4610-b4c5-a11353e8025d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:53.101565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4b30411b-40c6-4610-b4c5-a11353e8025d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:53.117638 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b30411b-40c6-4610-b4c5-a11353e8025d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:53.117638 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b30411b-40c6-4610-b4c5-a11353e8025d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:53.117638 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b30411b-40c6-4610-b4c5-a11353e8025d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:53.117638 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b30411b-40c6-4610-b4c5-a11353e8025d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:53.118385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b30411b-40c6-4610-b4c5-a11353e8025d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:53.118385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b30411b-40c6-4610-b4c5-a11353e8025d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:53.161349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4b30411b-40c6-4610-b4c5-a11353e8025d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.071432 Aug 07 01:19:53.161946 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 488/1005] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:53 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:53.596181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6ba20b1d-1672-4ab8-a4b8-212f93e7ac03 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:53.597704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6ba20b1d-1672-4ab8-a4b8-212f93e7ac03 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:53.608765 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba20b1d-1672-4ab8-a4b8-212f93e7ac03 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:53.609413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba20b1d-1672-4ab8-a4b8-212f93e7ac03 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:53.609937 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba20b1d-1672-4ab8-a4b8-212f93e7ac03 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:53.611001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba20b1d-1672-4ab8-a4b8-212f93e7ac03 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:53.611699 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba20b1d-1672-4ab8-a4b8-212f93e7ac03 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:53.612303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba20b1d-1672-4ab8-a4b8-212f93e7ac03 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:53.709946 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6ba20b1d-1672-4ab8-a4b8-212f93e7ac03 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:53.723900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6ba20b1d-1672-4ab8-a4b8-212f93e7ac03 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.133037 Aug 07 01:19:53.724874 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 518/1006] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:53 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:54.175041 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c5435617-c94b-4918-8ea4-d984ffeb451d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:54.177380 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c5435617-c94b-4918-8ea4-d984ffeb451d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:54.184606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c5435617-c94b-4918-8ea4-d984ffeb451d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:54.185094 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c5435617-c94b-4918-8ea4-d984ffeb451d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:54.185537 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c5435617-c94b-4918-8ea4-d984ffeb451d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:54.186345 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c5435617-c94b-4918-8ea4-d984ffeb451d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:54.186806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c5435617-c94b-4918-8ea4-d984ffeb451d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:54.187237 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c5435617-c94b-4918-8ea4-d984ffeb451d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:54.193761 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:19:54.229187 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c5435617-c94b-4918-8ea4-d984ffeb451d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057898 Aug 07 01:19:54.229463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 489/1007] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:54 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:54.760521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-242a2529-3b3f-455f-a815-2ed6dee73df1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:54.761684 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-242a2529-3b3f-455f-a815-2ed6dee73df1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:54.784702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-242a2529-3b3f-455f-a815-2ed6dee73df1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:54.785569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-242a2529-3b3f-455f-a815-2ed6dee73df1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:54.786326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-242a2529-3b3f-455f-a815-2ed6dee73df1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:54.787808 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-242a2529-3b3f-455f-a815-2ed6dee73df1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:54.788660 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-242a2529-3b3f-455f-a815-2ed6dee73df1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:54.789432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-242a2529-3b3f-455f-a815-2ed6dee73df1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:54.891985 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-242a2529-3b3f-455f-a815-2ed6dee73df1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:54.906726 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-242a2529-3b3f-455f-a815-2ed6dee73df1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.151256 Aug 07 01:19:54.907789 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 519/1008] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:54 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:19:55.244398 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f816ac21-21f5-48db-b644-f34d0832ca20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:55.247178 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f816ac21-21f5-48db-b644-f34d0832ca20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:55.255187 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f816ac21-21f5-48db-b644-f34d0832ca20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:55.255682 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f816ac21-21f5-48db-b644-f34d0832ca20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:55.256121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f816ac21-21f5-48db-b644-f34d0832ca20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:55.256928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f816ac21-21f5-48db-b644-f34d0832ca20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:55.257394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f816ac21-21f5-48db-b644-f34d0832ca20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:55.258384 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f816ac21-21f5-48db-b644-f34d0832ca20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:55.297061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f816ac21-21f5-48db-b644-f34d0832ca20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058273 Aug 07 01:19:55.297664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 490/1009] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:55 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:19:55.949334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6d1eb9f0-ee82-4bcd-97c1-ed65817c23f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:55.950656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6d1eb9f0-ee82-4bcd-97c1-ed65817c23f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:55.963603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d1eb9f0-ee82-4bcd-97c1-ed65817c23f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:55.964582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d1eb9f0-ee82-4bcd-97c1-ed65817c23f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:55.965370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d1eb9f0-ee82-4bcd-97c1-ed65817c23f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:55.966711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d1eb9f0-ee82-4bcd-97c1-ed65817c23f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:55.967555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d1eb9f0-ee82-4bcd-97c1-ed65817c23f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:55.968389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d1eb9f0-ee82-4bcd-97c1-ed65817c23f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:56.175731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6d1eb9f0-ee82-4bcd-97c1-ed65817c23f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:56.189905 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6d1eb9f0-ee82-4bcd-97c1-ed65817c23f1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.248552 Aug 07 01:19:56.191002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 520/1010] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:55 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:56.315345 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b91e4ce7-804a-44dd-b01d-929b097c2763 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:56.318881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b91e4ce7-804a-44dd-b01d-929b097c2763 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:56.331436 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b91e4ce7-804a-44dd-b01d-929b097c2763 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:56.332336 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b91e4ce7-804a-44dd-b01d-929b097c2763 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:56.333237 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b91e4ce7-804a-44dd-b01d-929b097c2763 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:56.338678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b91e4ce7-804a-44dd-b01d-929b097c2763 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:56.338678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b91e4ce7-804a-44dd-b01d-929b097c2763 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:56.338678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b91e4ce7-804a-44dd-b01d-929b097c2763 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:56.400025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b91e4ce7-804a-44dd-b01d-929b097c2763 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.092028 Aug 07 01:19:56.401168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 491/1011] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:56 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:57.226900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d9a25aba-350d-4801-ae84-949f4007862a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:57.227667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d9a25aba-350d-4801-ae84-949f4007862a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:57.235012 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9a25aba-350d-4801-ae84-949f4007862a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:57.235581 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9a25aba-350d-4801-ae84-949f4007862a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:57.236101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9a25aba-350d-4801-ae84-949f4007862a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:57.236877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9a25aba-350d-4801-ae84-949f4007862a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:57.237561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9a25aba-350d-4801-ae84-949f4007862a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:57.237997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9a25aba-350d-4801-ae84-949f4007862a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:57.334701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d9a25aba-350d-4801-ae84-949f4007862a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:57.344510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d9a25aba-350d-4801-ae84-949f4007862a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.122549 Aug 07 01:19:57.345111 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 521/1012] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:57 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:57.415911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f0907197-1781-4ca3-8329-6c109dff2661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:57.418600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f0907197-1781-4ca3-8329-6c109dff2661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:57.425867 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0907197-1781-4ca3-8329-6c109dff2661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:57.426373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0907197-1781-4ca3-8329-6c109dff2661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:57.426853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0907197-1781-4ca3-8329-6c109dff2661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:57.427660 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0907197-1781-4ca3-8329-6c109dff2661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:57.428130 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0907197-1781-4ca3-8329-6c109dff2661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:57.428606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0907197-1781-4ca3-8329-6c109dff2661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:57.465028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f0907197-1781-4ca3-8329-6c109dff2661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053288 Aug 07 01:19:57.465641 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 492/1013] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:57 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:58.367883 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-306a3f7a-6522-46af-ace1-7e8336bafa9c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:58.368918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-306a3f7a-6522-46af-ace1-7e8336bafa9c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:58.376981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-306a3f7a-6522-46af-ace1-7e8336bafa9c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:58.377682 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-306a3f7a-6522-46af-ace1-7e8336bafa9c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:58.378231 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-306a3f7a-6522-46af-ace1-7e8336bafa9c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:58.379239 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-306a3f7a-6522-46af-ace1-7e8336bafa9c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:58.379864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-306a3f7a-6522-46af-ace1-7e8336bafa9c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:58.380452 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-306a3f7a-6522-46af-ace1-7e8336bafa9c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:58.474794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-306a3f7a-6522-46af-ace1-7e8336bafa9c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:58.482001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-68e3a050-8167-423e-a180-6ad962d225f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:58.485904 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-68e3a050-8167-423e-a180-6ad962d225f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:58.489835 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-306a3f7a-6522-46af-ace1-7e8336bafa9c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.125752 Aug 07 01:19:58.490930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 522/1014] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:58 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:58.498097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-68e3a050-8167-423e-a180-6ad962d225f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:58.498966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-68e3a050-8167-423e-a180-6ad962d225f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:58.499728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-68e3a050-8167-423e-a180-6ad962d225f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:58.501011 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-68e3a050-8167-423e-a180-6ad962d225f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:58.501804 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-68e3a050-8167-423e-a180-6ad962d225f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:58.502601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-68e3a050-8167-423e-a180-6ad962d225f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:58.559068 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-68e3a050-8167-423e-a180-6ad962d225f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.084338 Aug 07 01:19:58.560017 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 493/1015] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:58 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:19:59.525141 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ab49e4c9-dd6d-4940-865c-5cecc563f141 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:59.526096 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ab49e4c9-dd6d-4940-865c-5cecc563f141 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:59.533296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab49e4c9-dd6d-4940-865c-5cecc563f141 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:59.533807 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab49e4c9-dd6d-4940-865c-5cecc563f141 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:59.534257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab49e4c9-dd6d-4940-865c-5cecc563f141 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:59.534993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab49e4c9-dd6d-4940-865c-5cecc563f141 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:59.535452 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab49e4c9-dd6d-4940-865c-5cecc563f141 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:59.535982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab49e4c9-dd6d-4940-865c-5cecc563f141 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:59.576839 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-016a777d-fc76-469e-bc7c-d532c14f879e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:19:59.580914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-016a777d-fc76-469e-bc7c-d532c14f879e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:19:59.592957 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-016a777d-fc76-469e-bc7c-d532c14f879e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:59.593534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-016a777d-fc76-469e-bc7c-d532c14f879e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:59.594096 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-016a777d-fc76-469e-bc7c-d532c14f879e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:59.594967 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-016a777d-fc76-469e-bc7c-d532c14f879e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:19:59.595565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-016a777d-fc76-469e-bc7c-d532c14f879e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:19:59.596086 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-016a777d-fc76-469e-bc7c-d532c14f879e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:19:59.640300 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ab49e4c9-dd6d-4940-865c-5cecc563f141 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:19:59.649075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ab49e4c9-dd6d-4940-865c-5cecc563f141 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.127809 Aug 07 01:19:59.649966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 523/1016] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:19:59 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:19:59.652839 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-016a777d-fc76-469e-bc7c-d532c14f879e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080359 Aug 07 01:19:59.653826 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 494/1017] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:19:59 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:00.668169 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a8f8cb94-56e9-4bae-9b3a-78974efbe9cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:00.670905 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a8f8cb94-56e9-4bae-9b3a-78974efbe9cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:00.678418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8f8cb94-56e9-4bae-9b3a-78974efbe9cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:00.679077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8f8cb94-56e9-4bae-9b3a-78974efbe9cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:00.680115 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8f8cb94-56e9-4bae-9b3a-78974efbe9cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:00.680897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8f8cb94-56e9-4bae-9b3a-78974efbe9cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:00.681364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8f8cb94-56e9-4bae-9b3a-78974efbe9cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:00.681795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8f8cb94-56e9-4bae-9b3a-78974efbe9cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:00.684646 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e2147568-da4d-468b-9d49-58346edc3952 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:00.685444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e2147568-da4d-468b-9d49-58346edc3952 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:00.692896 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2147568-da4d-468b-9d49-58346edc3952 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:00.693366 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2147568-da4d-468b-9d49-58346edc3952 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:00.693811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2147568-da4d-468b-9d49-58346edc3952 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:00.694536 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2147568-da4d-468b-9d49-58346edc3952 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:00.695733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2147568-da4d-468b-9d49-58346edc3952 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:00.696380 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2147568-da4d-468b-9d49-58346edc3952 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:00.746760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a8f8cb94-56e9-4bae-9b3a-78974efbe9cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.083218 Aug 07 01:20:00.747549 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 524/1018] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:00 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:00.823433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e2147568-da4d-468b-9d49-58346edc3952 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:00.832644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e2147568-da4d-468b-9d49-58346edc3952 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.153037 Aug 07 01:20:00.833818 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 495/1019] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:00 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:01.761857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-50091d58-f82e-487c-a482-8d67ad56454b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:01.765736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-50091d58-f82e-487c-a482-8d67ad56454b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:01.768964 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=54 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:20:01.775941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-50091d58-f82e-487c-a482-8d67ad56454b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:01.776477 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-50091d58-f82e-487c-a482-8d67ad56454b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:01.776936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-50091d58-f82e-487c-a482-8d67ad56454b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:01.778428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-50091d58-f82e-487c-a482-8d67ad56454b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:01.778930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-50091d58-f82e-487c-a482-8d67ad56454b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:01.779548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-50091d58-f82e-487c-a482-8d67ad56454b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:01.816451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-50091d58-f82e-487c-a482-8d67ad56454b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059010 Aug 07 01:20:01.817008 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 525/1020] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:01 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:01.859066 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-52175672-28fe-4599-bd87-c3689e6aab36 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:01.860354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-52175672-28fe-4599-bd87-c3689e6aab36 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:01.872760 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-52175672-28fe-4599-bd87-c3689e6aab36 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:01.873733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-52175672-28fe-4599-bd87-c3689e6aab36 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:01.874664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-52175672-28fe-4599-bd87-c3689e6aab36 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:01.876173 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-52175672-28fe-4599-bd87-c3689e6aab36 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:01.877324 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-52175672-28fe-4599-bd87-c3689e6aab36 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:01.878211 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-52175672-28fe-4599-bd87-c3689e6aab36 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:01.974997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-52175672-28fe-4599-bd87-c3689e6aab36 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:01.983211 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-52175672-28fe-4599-bd87-c3689e6aab36 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.132230 Aug 07 01:20:01.983814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 496/1021] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:01 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:02.829621 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9d7e6364-d110-46e6-ad78-f1ef4d5c579c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:02.831862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9d7e6364-d110-46e6-ad78-f1ef4d5c579c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:02.839748 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d7e6364-d110-46e6-ad78-f1ef4d5c579c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:02.840278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d7e6364-d110-46e6-ad78-f1ef4d5c579c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:02.840735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d7e6364-d110-46e6-ad78-f1ef4d5c579c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:02.841471 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d7e6364-d110-46e6-ad78-f1ef4d5c579c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:02.841936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d7e6364-d110-46e6-ad78-f1ef4d5c579c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:02.842398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d7e6364-d110-46e6-ad78-f1ef4d5c579c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:02.885360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9d7e6364-d110-46e6-ad78-f1ef4d5c579c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059943 Aug 07 01:20:02.886048 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 526/1022] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:02 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:03.006122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-84128610-56a7-4da7-9391-b8e5db000f92 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:03.006777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-84128610-56a7-4da7-9391-b8e5db000f92 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:03.010022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=56 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:20:03.016149 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84128610-56a7-4da7-9391-b8e5db000f92 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:03.016830 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84128610-56a7-4da7-9391-b8e5db000f92 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:03.017538 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84128610-56a7-4da7-9391-b8e5db000f92 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:03.018379 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84128610-56a7-4da7-9391-b8e5db000f92 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:03.018849 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84128610-56a7-4da7-9391-b8e5db000f92 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:03.019289 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84128610-56a7-4da7-9391-b8e5db000f92 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:03.097989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-84128610-56a7-4da7-9391-b8e5db000f92 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:03.110362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-84128610-56a7-4da7-9391-b8e5db000f92 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.108756 Aug 07 01:20:03.111428 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 497/1023] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:02 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:03.899820 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8a022668-b35f-4b36-9aa0-424c46027667 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:03.902494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8a022668-b35f-4b36-9aa0-424c46027667 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:03.910147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a022668-b35f-4b36-9aa0-424c46027667 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:03.910710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a022668-b35f-4b36-9aa0-424c46027667 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:03.911418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a022668-b35f-4b36-9aa0-424c46027667 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:03.912570 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a022668-b35f-4b36-9aa0-424c46027667 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:03.913265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a022668-b35f-4b36-9aa0-424c46027667 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:03.914412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a022668-b35f-4b36-9aa0-424c46027667 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:03.953873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8a022668-b35f-4b36-9aa0-424c46027667 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058634 Aug 07 01:20:03.954834 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 527/1024] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:03 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:04.132095 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d7e5cb31-f3dc-4ca3-ab1b-a9609045726a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:04.133226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d7e5cb31-f3dc-4ca3-ab1b-a9609045726a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:04.141769 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7e5cb31-f3dc-4ca3-ab1b-a9609045726a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:04.142269 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7e5cb31-f3dc-4ca3-ab1b-a9609045726a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:04.142729 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7e5cb31-f3dc-4ca3-ab1b-a9609045726a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:04.143619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7e5cb31-f3dc-4ca3-ab1b-a9609045726a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:04.144086 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7e5cb31-f3dc-4ca3-ab1b-a9609045726a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:04.144563 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7e5cb31-f3dc-4ca3-ab1b-a9609045726a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:04.221012 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d7e5cb31-f3dc-4ca3-ab1b-a9609045726a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:04.235464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d7e5cb31-f3dc-4ca3-ab1b-a9609045726a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.106956 Aug 07 01:20:04.236581 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 498/1025] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:04 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:04.972684 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5973958c-b9e8-463d-84f6-831d03b942fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:04.976679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5973958c-b9e8-463d-84f6-831d03b942fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:04.988441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5973958c-b9e8-463d-84f6-831d03b942fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:04.989355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5973958c-b9e8-463d-84f6-831d03b942fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:04.990184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5973958c-b9e8-463d-84f6-831d03b942fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:04.992287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5973958c-b9e8-463d-84f6-831d03b942fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:04.993216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5973958c-b9e8-463d-84f6-831d03b942fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:04.994011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5973958c-b9e8-463d-84f6-831d03b942fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:05.056686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5973958c-b9e8-463d-84f6-831d03b942fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.090569 Aug 07 01:20:05.057795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 528/1026] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:04 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:05.263281 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3fac8afd-dc4f-4f4d-bcfc-140479033497 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:05.263982 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3fac8afd-dc4f-4f4d-bcfc-140479033497 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:05.276986 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fac8afd-dc4f-4f4d-bcfc-140479033497 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:05.277881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fac8afd-dc4f-4f4d-bcfc-140479033497 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:05.278768 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fac8afd-dc4f-4f4d-bcfc-140479033497 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:05.279972 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fac8afd-dc4f-4f4d-bcfc-140479033497 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:05.280951 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fac8afd-dc4f-4f4d-bcfc-140479033497 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:05.282330 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fac8afd-dc4f-4f4d-bcfc-140479033497 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:05.355952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3fac8afd-dc4f-4f4d-bcfc-140479033497 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:05.370415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3fac8afd-dc4f-4f4d-bcfc-140479033497 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.114244 Aug 07 01:20:05.371493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 499/1027] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:05 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:06.072960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-eaecd620-c2e4-48cd-b3ac-2e93372061c8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:06.075538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-eaecd620-c2e4-48cd-b3ac-2e93372061c8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:06.086379 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eaecd620-c2e4-48cd-b3ac-2e93372061c8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:06.086947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eaecd620-c2e4-48cd-b3ac-2e93372061c8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:06.087438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eaecd620-c2e4-48cd-b3ac-2e93372061c8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:06.088269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eaecd620-c2e4-48cd-b3ac-2e93372061c8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:06.088867 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eaecd620-c2e4-48cd-b3ac-2e93372061c8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:06.089399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eaecd620-c2e4-48cd-b3ac-2e93372061c8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:06.124647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-eaecd620-c2e4-48cd-b3ac-2e93372061c8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055574 Aug 07 01:20:06.125336 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 529/1028] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:06 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:06.397845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-73683c02-2779-454a-ba9f-2c19bae1806a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:06.399244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-73683c02-2779-454a-ba9f-2c19bae1806a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:06.406930 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-73683c02-2779-454a-ba9f-2c19bae1806a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:06.407617 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-73683c02-2779-454a-ba9f-2c19bae1806a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:06.408104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-73683c02-2779-454a-ba9f-2c19bae1806a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:06.408928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-73683c02-2779-454a-ba9f-2c19bae1806a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:06.409432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-73683c02-2779-454a-ba9f-2c19bae1806a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:06.409897 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-73683c02-2779-454a-ba9f-2c19bae1806a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:06.489408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-73683c02-2779-454a-ba9f-2c19bae1806a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:06.500316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-73683c02-2779-454a-ba9f-2c19bae1806a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.109076 Aug 07 01:20:06.500986 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 500/1029] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:06 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:07.143494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dcbeb424-4884-49f0-8990-eda739f31802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:07.147370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dcbeb424-4884-49f0-8990-eda739f31802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:07.160024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dcbeb424-4884-49f0-8990-eda739f31802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:07.160921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dcbeb424-4884-49f0-8990-eda739f31802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:07.161729 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dcbeb424-4884-49f0-8990-eda739f31802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:07.163047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dcbeb424-4884-49f0-8990-eda739f31802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:07.163845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dcbeb424-4884-49f0-8990-eda739f31802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:07.163845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dcbeb424-4884-49f0-8990-eda739f31802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:07.198418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:20:07.226571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dcbeb424-4884-49f0-8990-eda739f31802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.090650 Aug 07 01:20:07.227637 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 530/1030] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:07 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:07.531097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-84cc255a-d12f-45a4-979c-5956c9c1a8a8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:07.531900 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-84cc255a-d12f-45a4-979c-5956c9c1a8a8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:07.540315 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84cc255a-d12f-45a4-979c-5956c9c1a8a8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:07.540687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84cc255a-d12f-45a4-979c-5956c9c1a8a8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:07.541276 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84cc255a-d12f-45a4-979c-5956c9c1a8a8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:07.542135 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84cc255a-d12f-45a4-979c-5956c9c1a8a8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:07.542642 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84cc255a-d12f-45a4-979c-5956c9c1a8a8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:07.543285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84cc255a-d12f-45a4-979c-5956c9c1a8a8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:07.616012 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-84cc255a-d12f-45a4-979c-5956c9c1a8a8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:07.625915 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-84cc255a-d12f-45a4-979c-5956c9c1a8a8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.098766 Aug 07 01:20:07.626513 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 501/1031] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:07 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:07.983884 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:20:07.984198 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:20:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:20:08.241748 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fc30923f-bedd-45a6-b1b0-b8427e53f297 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:08.244494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fc30923f-bedd-45a6-b1b0-b8427e53f297 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:08.251898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc30923f-bedd-45a6-b1b0-b8427e53f297 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:08.252424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc30923f-bedd-45a6-b1b0-b8427e53f297 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:08.252862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc30923f-bedd-45a6-b1b0-b8427e53f297 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:08.253584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc30923f-bedd-45a6-b1b0-b8427e53f297 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:08.254035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc30923f-bedd-45a6-b1b0-b8427e53f297 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:08.254462 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc30923f-bedd-45a6-b1b0-b8427e53f297 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:08.288906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fc30923f-bedd-45a6-b1b0-b8427e53f297 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051582 Aug 07 01:20:08.289528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 531/1032] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:08 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:08.662178 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0c1b9e5d-5a53-42f6-aa53-05d2a87477e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:08.663124 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0c1b9e5d-5a53-42f6-aa53-05d2a87477e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:08.670347 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0c1b9e5d-5a53-42f6-aa53-05d2a87477e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:08.670920 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0c1b9e5d-5a53-42f6-aa53-05d2a87477e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:08.671540 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0c1b9e5d-5a53-42f6-aa53-05d2a87477e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:08.672574 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0c1b9e5d-5a53-42f6-aa53-05d2a87477e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:08.673171 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0c1b9e5d-5a53-42f6-aa53-05d2a87477e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:08.673694 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0c1b9e5d-5a53-42f6-aa53-05d2a87477e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:08.750667 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0c1b9e5d-5a53-42f6-aa53-05d2a87477e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:08.760285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0c1b9e5d-5a53-42f6-aa53-05d2a87477e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.102291 Aug 07 01:20:08.760948 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 502/1033] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:08 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:09.301855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5c36cc03-26a3-49ea-938e-c93c3815bac8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:09.305797 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5c36cc03-26a3-49ea-938e-c93c3815bac8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:09.316026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5c36cc03-26a3-49ea-938e-c93c3815bac8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:09.316553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5c36cc03-26a3-49ea-938e-c93c3815bac8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:09.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5c36cc03-26a3-49ea-938e-c93c3815bac8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:09.317740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5c36cc03-26a3-49ea-938e-c93c3815bac8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:09.318193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5c36cc03-26a3-49ea-938e-c93c3815bac8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:09.318621 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5c36cc03-26a3-49ea-938e-c93c3815bac8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:09.378453 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5c36cc03-26a3-49ea-938e-c93c3815bac8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080428 Aug 07 01:20:09.379173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 532/1034] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:09 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:09.782192 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-aa2254f7-377e-4ae6-a5e8-b5275c4af32b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:09.782849 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-aa2254f7-377e-4ae6-a5e8-b5275c4af32b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:09.790771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aa2254f7-377e-4ae6-a5e8-b5275c4af32b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:09.791427 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aa2254f7-377e-4ae6-a5e8-b5275c4af32b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:09.791872 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aa2254f7-377e-4ae6-a5e8-b5275c4af32b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:09.792648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aa2254f7-377e-4ae6-a5e8-b5275c4af32b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:09.793166 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aa2254f7-377e-4ae6-a5e8-b5275c4af32b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:09.793597 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aa2254f7-377e-4ae6-a5e8-b5275c4af32b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:09.862447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-aa2254f7-377e-4ae6-a5e8-b5275c4af32b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:09.871974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-aa2254f7-377e-4ae6-a5e8-b5275c4af32b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.093643 Aug 07 01:20:09.872685 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 503/1035] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:09 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:10.398948 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7d4356c7-18dd-4f6a-b960-c25f83b49296 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:10.402877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7d4356c7-18dd-4f6a-b960-c25f83b49296 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:10.414868 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d4356c7-18dd-4f6a-b960-c25f83b49296 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:10.415756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d4356c7-18dd-4f6a-b960-c25f83b49296 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:10.416628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d4356c7-18dd-4f6a-b960-c25f83b49296 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:10.417910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d4356c7-18dd-4f6a-b960-c25f83b49296 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:10.418754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d4356c7-18dd-4f6a-b960-c25f83b49296 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:10.419574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d4356c7-18dd-4f6a-b960-c25f83b49296 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:10.482273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7d4356c7-18dd-4f6a-b960-c25f83b49296 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.091992 Aug 07 01:20:10.483481 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 533/1036] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:10 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:10.893117 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-dda2cfb5-8db9-4703-b0fa-2bbf28122c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:10.893764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-dda2cfb5-8db9-4703-b0fa-2bbf28122c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:10.900829 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dda2cfb5-8db9-4703-b0fa-2bbf28122c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:10.901329 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dda2cfb5-8db9-4703-b0fa-2bbf28122c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:10.901768 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dda2cfb5-8db9-4703-b0fa-2bbf28122c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:10.902496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dda2cfb5-8db9-4703-b0fa-2bbf28122c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:10.902948 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dda2cfb5-8db9-4703-b0fa-2bbf28122c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:10.903375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dda2cfb5-8db9-4703-b0fa-2bbf28122c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:10.975151 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dda2cfb5-8db9-4703-b0fa-2bbf28122c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:10.984798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-dda2cfb5-8db9-4703-b0fa-2bbf28122c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.095772 Aug 07 01:20:10.985454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 504/1037] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:10 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:11.499914 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-780d530f-1df4-41e5-9852-a9a68cc413da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:11.503045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-780d530f-1df4-41e5-9852-a9a68cc413da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:11.512151 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-780d530f-1df4-41e5-9852-a9a68cc413da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:11.512731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-780d530f-1df4-41e5-9852-a9a68cc413da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:11.513213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-780d530f-1df4-41e5-9852-a9a68cc413da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:11.514063 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-780d530f-1df4-41e5-9852-a9a68cc413da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:11.514568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-780d530f-1df4-41e5-9852-a9a68cc413da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:11.515007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-780d530f-1df4-41e5-9852-a9a68cc413da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:11.557955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-780d530f-1df4-41e5-9852-a9a68cc413da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062454 Aug 07 01:20:11.558986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 534/1038] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:11 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:12.010495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bcf05347-5129-4278-ad7d-4231cbdcf0d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:12.011502 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bcf05347-5129-4278-ad7d-4231cbdcf0d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:12.024703 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bcf05347-5129-4278-ad7d-4231cbdcf0d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:12.025271 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bcf05347-5129-4278-ad7d-4231cbdcf0d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:12.025703 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bcf05347-5129-4278-ad7d-4231cbdcf0d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:12.026518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bcf05347-5129-4278-ad7d-4231cbdcf0d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:12.027032 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bcf05347-5129-4278-ad7d-4231cbdcf0d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:12.027528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bcf05347-5129-4278-ad7d-4231cbdcf0d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:12.132872 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bcf05347-5129-4278-ad7d-4231cbdcf0d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:12.140103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:20:12.151219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bcf05347-5129-4278-ad7d-4231cbdcf0d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.147641 Aug 07 01:20:12.152299 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 505/1039] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:12 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:12.579431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5cd84e4d-73d4-4453-9a19-f4df7ac05cf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:12.583286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5cd84e4d-73d4-4453-9a19-f4df7ac05cf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:12.595919 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5cd84e4d-73d4-4453-9a19-f4df7ac05cf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:12.596818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5cd84e4d-73d4-4453-9a19-f4df7ac05cf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:12.597601 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5cd84e4d-73d4-4453-9a19-f4df7ac05cf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:12.598919 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5cd84e4d-73d4-4453-9a19-f4df7ac05cf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:12.599759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5cd84e4d-73d4-4453-9a19-f4df7ac05cf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:12.600600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5cd84e4d-73d4-4453-9a19-f4df7ac05cf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:12.662330 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5cd84e4d-73d4-4453-9a19-f4df7ac05cf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.090848 Aug 07 01:20:12.663975 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 535/1040] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:12 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:13.179739 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-26726353-2ac3-4ff3-ac8c-cf254c43e50b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:13.180406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-26726353-2ac3-4ff3-ac8c-cf254c43e50b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:13.188650 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26726353-2ac3-4ff3-ac8c-cf254c43e50b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:13.189261 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26726353-2ac3-4ff3-ac8c-cf254c43e50b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:13.189745 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26726353-2ac3-4ff3-ac8c-cf254c43e50b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:13.190686 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26726353-2ac3-4ff3-ac8c-cf254c43e50b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:13.191213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26726353-2ac3-4ff3-ac8c-cf254c43e50b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:13.191682 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26726353-2ac3-4ff3-ac8c-cf254c43e50b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:13.285723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-26726353-2ac3-4ff3-ac8c-cf254c43e50b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:13.295888 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-26726353-2ac3-4ff3-ac8c-cf254c43e50b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.118928 Aug 07 01:20:13.295888 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 506/1041] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:13 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:13.677779 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-efebbc21-e636-48f7-b41e-e467a1cc3027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:13.681090 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-efebbc21-e636-48f7-b41e-e467a1cc3027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:13.688899 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-efebbc21-e636-48f7-b41e-e467a1cc3027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:13.689397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-efebbc21-e636-48f7-b41e-e467a1cc3027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:13.690418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-efebbc21-e636-48f7-b41e-e467a1cc3027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:13.690867 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-efebbc21-e636-48f7-b41e-e467a1cc3027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:13.691169 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-efebbc21-e636-48f7-b41e-e467a1cc3027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:13.691605 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-efebbc21-e636-48f7-b41e-e467a1cc3027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:13.729061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-efebbc21-e636-48f7-b41e-e467a1cc3027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055988 Aug 07 01:20:13.729663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 536/1042] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:13 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:14.174505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:20:14.174807 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:20:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:20:14.323621 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fe61f82f-12b3-427c-9026-90d1b9d9c431 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:14.324535 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fe61f82f-12b3-427c-9026-90d1b9d9c431 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:14.332682 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe61f82f-12b3-427c-9026-90d1b9d9c431 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:14.333316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe61f82f-12b3-427c-9026-90d1b9d9c431 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:14.333619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe61f82f-12b3-427c-9026-90d1b9d9c431 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:14.334442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe61f82f-12b3-427c-9026-90d1b9d9c431 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:14.335623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe61f82f-12b3-427c-9026-90d1b9d9c431 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:14.336053 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe61f82f-12b3-427c-9026-90d1b9d9c431 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:14.430726 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fe61f82f-12b3-427c-9026-90d1b9d9c431 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:14.445016 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fe61f82f-12b3-427c-9026-90d1b9d9c431 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.126675 Aug 07 01:20:14.446099 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 507/1043] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:14 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:14.747850 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8bd23e77-1382-4d3f-939b-749f762e47ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:14.750685 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8bd23e77-1382-4d3f-939b-749f762e47ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:14.759718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8bd23e77-1382-4d3f-939b-749f762e47ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:14.760305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8bd23e77-1382-4d3f-939b-749f762e47ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:14.760817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8bd23e77-1382-4d3f-939b-749f762e47ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:14.761657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8bd23e77-1382-4d3f-939b-749f762e47ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:14.762250 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8bd23e77-1382-4d3f-939b-749f762e47ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:14.762733 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8bd23e77-1382-4d3f-939b-749f762e47ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:14.815457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8bd23e77-1382-4d3f-939b-749f762e47ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.072563 Aug 07 01:20:14.816649 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 537/1044] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:14 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:15.479576 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-365211d8-e9c1-4a69-b408-89dfcfee9d35 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:15.480394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-365211d8-e9c1-4a69-b408-89dfcfee9d35 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:15.487731 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-365211d8-e9c1-4a69-b408-89dfcfee9d35 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:15.488436 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-365211d8-e9c1-4a69-b408-89dfcfee9d35 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:15.488932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-365211d8-e9c1-4a69-b408-89dfcfee9d35 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:15.489726 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-365211d8-e9c1-4a69-b408-89dfcfee9d35 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:15.490354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-365211d8-e9c1-4a69-b408-89dfcfee9d35 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:15.490838 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-365211d8-e9c1-4a69-b408-89dfcfee9d35 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:15.565524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-365211d8-e9c1-4a69-b408-89dfcfee9d35 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:15.574877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-365211d8-e9c1-4a69-b408-89dfcfee9d35 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.099453 Aug 07 01:20:15.575820 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 508/1045] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:15 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:15.832057 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b307a2be-42bb-4efc-8036-93b1cec7aa54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:15.834392 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b307a2be-42bb-4efc-8036-93b1cec7aa54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:15.841781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b307a2be-42bb-4efc-8036-93b1cec7aa54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:15.842274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b307a2be-42bb-4efc-8036-93b1cec7aa54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:15.842719 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b307a2be-42bb-4efc-8036-93b1cec7aa54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:15.843481 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b307a2be-42bb-4efc-8036-93b1cec7aa54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:15.844099 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b307a2be-42bb-4efc-8036-93b1cec7aa54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:15.844564 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b307a2be-42bb-4efc-8036-93b1cec7aa54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:15.887407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b307a2be-42bb-4efc-8036-93b1cec7aa54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059574 Aug 07 01:20:15.888006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 538/1046] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:15 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:16.597585 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-780c134e-bee2-4888-8d78-a611c21d5c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:16.598227 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-780c134e-bee2-4888-8d78-a611c21d5c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:16.605518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-780c134e-bee2-4888-8d78-a611c21d5c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:16.606007 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-780c134e-bee2-4888-8d78-a611c21d5c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:16.606442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-780c134e-bee2-4888-8d78-a611c21d5c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:16.607212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-780c134e-bee2-4888-8d78-a611c21d5c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:16.607606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-780c134e-bee2-4888-8d78-a611c21d5c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:16.608027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-780c134e-bee2-4888-8d78-a611c21d5c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:16.684778 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-780c134e-bee2-4888-8d78-a611c21d5c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:16.692942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-780c134e-bee2-4888-8d78-a611c21d5c4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.099267 Aug 07 01:20:16.693714 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 509/1047] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:16 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:16.901769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cb9e56a2-651f-49bd-aada-84c7891290b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:16.903885 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cb9e56a2-651f-49bd-aada-84c7891290b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:16.910881 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cb9e56a2-651f-49bd-aada-84c7891290b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:16.911378 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cb9e56a2-651f-49bd-aada-84c7891290b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:16.911818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cb9e56a2-651f-49bd-aada-84c7891290b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:16.912579 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cb9e56a2-651f-49bd-aada-84c7891290b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:16.913659 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cb9e56a2-651f-49bd-aada-84c7891290b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:16.914099 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cb9e56a2-651f-49bd-aada-84c7891290b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:16.969361 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cb9e56a2-651f-49bd-aada-84c7891290b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.072334 Aug 07 01:20:16.970420 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 539/1048] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:16 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:17.715031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f6e73fa1-93ad-43ed-808d-6dec3710de51 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:17.715712 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f6e73fa1-93ad-43ed-808d-6dec3710de51 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:17.723335 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6e73fa1-93ad-43ed-808d-6dec3710de51 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:17.724060 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6e73fa1-93ad-43ed-808d-6dec3710de51 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:17.724982 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6e73fa1-93ad-43ed-808d-6dec3710de51 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:17.725795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6e73fa1-93ad-43ed-808d-6dec3710de51 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:17.726590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6e73fa1-93ad-43ed-808d-6dec3710de51 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:17.727267 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6e73fa1-93ad-43ed-808d-6dec3710de51 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:17.804504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f6e73fa1-93ad-43ed-808d-6dec3710de51 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:17.812985 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f6e73fa1-93ad-43ed-808d-6dec3710de51 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.102091 Aug 07 01:20:17.813572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 510/1049] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:17 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:17.983165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6c1171e9-cfbb-4762-bfd4-52a9b3b306ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:17.985508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6c1171e9-cfbb-4762-bfd4-52a9b3b306ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:17.993484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c1171e9-cfbb-4762-bfd4-52a9b3b306ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:17.994120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c1171e9-cfbb-4762-bfd4-52a9b3b306ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:17.994624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c1171e9-cfbb-4762-bfd4-52a9b3b306ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:17.995396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c1171e9-cfbb-4762-bfd4-52a9b3b306ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:17.996112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c1171e9-cfbb-4762-bfd4-52a9b3b306ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:17.996521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c1171e9-cfbb-4762-bfd4-52a9b3b306ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:18.034046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6c1171e9-cfbb-4762-bfd4-52a9b3b306ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055205 Aug 07 01:20:18.034720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 540/1050] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:17 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:18.837479 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7c6c7280-6481-43cc-83c7-c3220625abe3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:18.838146 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7c6c7280-6481-43cc-83c7-c3220625abe3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:18.845927 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7c6c7280-6481-43cc-83c7-c3220625abe3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:18.846428 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7c6c7280-6481-43cc-83c7-c3220625abe3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:18.846884 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7c6c7280-6481-43cc-83c7-c3220625abe3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:18.847628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7c6c7280-6481-43cc-83c7-c3220625abe3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:18.848089 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7c6c7280-6481-43cc-83c7-c3220625abe3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:18.848696 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7c6c7280-6481-43cc-83c7-c3220625abe3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:18.919254 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7c6c7280-6481-43cc-83c7-c3220625abe3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:18.927931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7c6c7280-6481-43cc-83c7-c3220625abe3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.095436 Aug 07 01:20:18.929428 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 511/1051] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:18 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:19.049074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-445379b8-f988-4b67-970c-220f32bf85f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:19.051428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-445379b8-f988-4b67-970c-220f32bf85f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:19.059253 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-445379b8-f988-4b67-970c-220f32bf85f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:19.059749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-445379b8-f988-4b67-970c-220f32bf85f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:19.060192 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-445379b8-f988-4b67-970c-220f32bf85f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:19.060956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-445379b8-f988-4b67-970c-220f32bf85f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:19.061412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-445379b8-f988-4b67-970c-220f32bf85f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:19.061917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-445379b8-f988-4b67-970c-220f32bf85f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:19.098351 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-445379b8-f988-4b67-970c-220f32bf85f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054264 Aug 07 01:20:19.098860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 541/1052] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:19 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:19.963004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a4a89b77-f097-4fc7-b0f9-0aaf0bfdd3dd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:19.963579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a4a89b77-f097-4fc7-b0f9-0aaf0bfdd3dd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:19.970479 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a4a89b77-f097-4fc7-b0f9-0aaf0bfdd3dd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:19.970972 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a4a89b77-f097-4fc7-b0f9-0aaf0bfdd3dd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:19.971414 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a4a89b77-f097-4fc7-b0f9-0aaf0bfdd3dd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:19.972133 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a4a89b77-f097-4fc7-b0f9-0aaf0bfdd3dd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:19.972620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a4a89b77-f097-4fc7-b0f9-0aaf0bfdd3dd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:19.973043 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a4a89b77-f097-4fc7-b0f9-0aaf0bfdd3dd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:20.068314 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a4a89b77-f097-4fc7-b0f9-0aaf0bfdd3dd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:20.077771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a4a89b77-f097-4fc7-b0f9-0aaf0bfdd3dd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.119537 Aug 07 01:20:20.078492 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 512/1053] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:19 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:20.114536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0cd673d6-6244-4e6b-8ddd-b7606ac349b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:20.118013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0cd673d6-6244-4e6b-8ddd-b7606ac349b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:20.127106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cd673d6-6244-4e6b-8ddd-b7606ac349b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:20.127611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cd673d6-6244-4e6b-8ddd-b7606ac349b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:20.128091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cd673d6-6244-4e6b-8ddd-b7606ac349b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:20.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cd673d6-6244-4e6b-8ddd-b7606ac349b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:20.129893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cd673d6-6244-4e6b-8ddd-b7606ac349b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:20.130484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cd673d6-6244-4e6b-8ddd-b7606ac349b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:20.173163 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0cd673d6-6244-4e6b-8ddd-b7606ac349b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063128 Aug 07 01:20:20.174528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 542/1054] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:20 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:21.101220 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-41d77619-26a5-4d0e-b88d-e85ff12a5733 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:21.102018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-41d77619-26a5-4d0e-b88d-e85ff12a5733 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:21.109509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41d77619-26a5-4d0e-b88d-e85ff12a5733 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:21.110036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41d77619-26a5-4d0e-b88d-e85ff12a5733 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:21.110598 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41d77619-26a5-4d0e-b88d-e85ff12a5733 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:21.111533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41d77619-26a5-4d0e-b88d-e85ff12a5733 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:21.112335 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41d77619-26a5-4d0e-b88d-e85ff12a5733 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:21.112953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41d77619-26a5-4d0e-b88d-e85ff12a5733 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:21.190369 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-41d77619-26a5-4d0e-b88d-e85ff12a5733 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:21.191318 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-19ada7fb-4289-4751-bf9f-109be93cca84 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:21.193465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-19ada7fb-4289-4751-bf9f-109be93cca84 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:21.201448 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19ada7fb-4289-4751-bf9f-109be93cca84 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:21.201949 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19ada7fb-4289-4751-bf9f-109be93cca84 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:21.202390 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19ada7fb-4289-4751-bf9f-109be93cca84 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:21.203131 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19ada7fb-4289-4751-bf9f-109be93cca84 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:21.203624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19ada7fb-4289-4751-bf9f-109be93cca84 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:21.204025 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-19ada7fb-4289-4751-bf9f-109be93cca84 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:21.205351 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-41d77619-26a5-4d0e-b88d-e85ff12a5733 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.107838 Aug 07 01:20:21.206441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 513/1055] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:21 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:21.241348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-19ada7fb-4289-4751-bf9f-109be93cca84 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054391 Aug 07 01:20:21.242058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 543/1056] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:21 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:22.229320 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-888215e5-438b-453a-a96d-9652928985cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:22.230204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-888215e5-438b-453a-a96d-9652928985cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:22.238044 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-888215e5-438b-453a-a96d-9652928985cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:22.238594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-888215e5-438b-453a-a96d-9652928985cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:22.239128 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-888215e5-438b-453a-a96d-9652928985cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:22.240001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-888215e5-438b-453a-a96d-9652928985cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:22.240562 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-888215e5-438b-453a-a96d-9652928985cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:22.241658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-888215e5-438b-453a-a96d-9652928985cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:22.259606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e771d3c1-56b5-44fb-9a2b-a63c04080140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:22.263498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e771d3c1-56b5-44fb-9a2b-a63c04080140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:22.275572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e771d3c1-56b5-44fb-9a2b-a63c04080140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:22.276472 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e771d3c1-56b5-44fb-9a2b-a63c04080140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:22.277209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e771d3c1-56b5-44fb-9a2b-a63c04080140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:22.278460 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e771d3c1-56b5-44fb-9a2b-a63c04080140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:22.279290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e771d3c1-56b5-44fb-9a2b-a63c04080140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:22.280053 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e771d3c1-56b5-44fb-9a2b-a63c04080140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:22.322800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-888215e5-438b-453a-a96d-9652928985cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:22.333653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-888215e5-438b-453a-a96d-9652928985cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.108122 Aug 07 01:20:22.333887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 514/1057] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:22 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:22.342198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e771d3c1-56b5-44fb-9a2b-a63c04080140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.090476 Aug 07 01:20:22.343364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 544/1058] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:22 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:23.357148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f62e80ca-ec44-4066-90b1-aa2c0fb0c55f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:23.357858 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f62e80ca-ec44-4066-90b1-aa2c0fb0c55f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:23.363156 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4c87273f-8439-4360-94dd-2ad4aa3da353 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:23.365961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f62e80ca-ec44-4066-90b1-aa2c0fb0c55f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:23.366551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f62e80ca-ec44-4066-90b1-aa2c0fb0c55f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:23.367082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4c87273f-8439-4360-94dd-2ad4aa3da353 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:23.367242 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f62e80ca-ec44-4066-90b1-aa2c0fb0c55f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:23.367939 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f62e80ca-ec44-4066-90b1-aa2c0fb0c55f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:23.368558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f62e80ca-ec44-4066-90b1-aa2c0fb0c55f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:23.369134 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f62e80ca-ec44-4066-90b1-aa2c0fb0c55f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:23.378628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4c87273f-8439-4360-94dd-2ad4aa3da353 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:23.379684 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4c87273f-8439-4360-94dd-2ad4aa3da353 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:23.380383 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4c87273f-8439-4360-94dd-2ad4aa3da353 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:23.382234 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4c87273f-8439-4360-94dd-2ad4aa3da353 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:23.383114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4c87273f-8439-4360-94dd-2ad4aa3da353 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:23.383919 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4c87273f-8439-4360-94dd-2ad4aa3da353 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:23.439859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f62e80ca-ec44-4066-90b1-aa2c0fb0c55f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:23.442672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4c87273f-8439-4360-94dd-2ad4aa3da353 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.087087 Aug 07 01:20:23.444465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 545/1059] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:23 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:23.453683 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f62e80ca-ec44-4066-90b1-aa2c0fb0c55f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100578 Aug 07 01:20:23.454413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 515/1060] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:23 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:24.465247 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0ad96b7c-266d-4713-a7c7-47042429ade1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:24.468544 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0ad96b7c-266d-4713-a7c7-47042429ade1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:24.475004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:20:24.478099 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:20:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments (ip 10.222.0.26) !!! Aug 07 01:20:24.479013 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f4d3cb14-06fd-4781-aeea-957bda0e0807 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:24.480010 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f4d3cb14-06fd-4781-aeea-957bda0e0807 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:24.480796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0ad96b7c-266d-4713-a7c7-47042429ade1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:24.482527 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0ad96b7c-266d-4713-a7c7-47042429ade1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:24.484421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0ad96b7c-266d-4713-a7c7-47042429ade1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:24.486614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0ad96b7c-266d-4713-a7c7-47042429ade1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:24.487573 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0ad96b7c-266d-4713-a7c7-47042429ade1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:24.488659 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0ad96b7c-266d-4713-a7c7-47042429ade1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:24.491331 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f4d3cb14-06fd-4781-aeea-957bda0e0807 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:24.492219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f4d3cb14-06fd-4781-aeea-957bda0e0807 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:24.493013 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f4d3cb14-06fd-4781-aeea-957bda0e0807 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:24.494261 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f4d3cb14-06fd-4781-aeea-957bda0e0807 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:24.495102 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f4d3cb14-06fd-4781-aeea-957bda0e0807 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:24.495878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f4d3cb14-06fd-4781-aeea-957bda0e0807 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:24.528201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0ad96b7c-266d-4713-a7c7-47042429ade1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.070609 Aug 07 01:20:24.529092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 546/1061] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:24 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:24.592986 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f4d3cb14-06fd-4781-aeea-957bda0e0807 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:24.602464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f4d3cb14-06fd-4781-aeea-957bda0e0807 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.129462 Aug 07 01:20:24.603235 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 516/1062] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:24 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:25.542453 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-17d0f0e4-b7de-425f-82d7-ee4be92897a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:25.544695 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-17d0f0e4-b7de-425f-82d7-ee4be92897a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:25.551603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17d0f0e4-b7de-425f-82d7-ee4be92897a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:25.552101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17d0f0e4-b7de-425f-82d7-ee4be92897a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:25.552577 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17d0f0e4-b7de-425f-82d7-ee4be92897a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:25.553944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17d0f0e4-b7de-425f-82d7-ee4be92897a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:25.555469 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17d0f0e4-b7de-425f-82d7-ee4be92897a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:25.555469 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17d0f0e4-b7de-425f-82d7-ee4be92897a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:25.560958 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:20:25.593302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-17d0f0e4-b7de-425f-82d7-ee4be92897a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055827 Aug 07 01:20:25.594100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 547/1063] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:25 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:25.623409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b4453dc9-7a5d-49b3-bbf9-de1f253ab7ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:25.624162 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b4453dc9-7a5d-49b3-bbf9-de1f253ab7ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:25.631510 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b4453dc9-7a5d-49b3-bbf9-de1f253ab7ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:25.632018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b4453dc9-7a5d-49b3-bbf9-de1f253ab7ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:25.632496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b4453dc9-7a5d-49b3-bbf9-de1f253ab7ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:25.633221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b4453dc9-7a5d-49b3-bbf9-de1f253ab7ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:25.633678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b4453dc9-7a5d-49b3-bbf9-de1f253ab7ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:25.634107 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b4453dc9-7a5d-49b3-bbf9-de1f253ab7ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:25.707598 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b4453dc9-7a5d-49b3-bbf9-de1f253ab7ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:25.716964 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b4453dc9-7a5d-49b3-bbf9-de1f253ab7ff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097554 Aug 07 01:20:25.717616 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 517/1064] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:25 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:26.607759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-aafb566a-a30c-4066-9d14-b8adb00bfd39 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:26.610113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-aafb566a-a30c-4066-9d14-b8adb00bfd39 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:26.619620 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aafb566a-a30c-4066-9d14-b8adb00bfd39 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:26.620783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aafb566a-a30c-4066-9d14-b8adb00bfd39 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:26.621488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aafb566a-a30c-4066-9d14-b8adb00bfd39 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:26.622353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aafb566a-a30c-4066-9d14-b8adb00bfd39 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:26.622832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aafb566a-a30c-4066-9d14-b8adb00bfd39 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:26.623271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aafb566a-a30c-4066-9d14-b8adb00bfd39 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:26.668679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-aafb566a-a30c-4066-9d14-b8adb00bfd39 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064667 Aug 07 01:20:26.669727 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 548/1065] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:26 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:26.749547 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c99241e5-4d80-4f88-9e09-373948da2717 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:26.750543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c99241e5-4d80-4f88-9e09-373948da2717 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:26.758429 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c99241e5-4d80-4f88-9e09-373948da2717 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:26.759029 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c99241e5-4d80-4f88-9e09-373948da2717 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:26.759343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c99241e5-4d80-4f88-9e09-373948da2717 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:26.760073 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c99241e5-4d80-4f88-9e09-373948da2717 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:26.760653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c99241e5-4d80-4f88-9e09-373948da2717 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:26.761120 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c99241e5-4d80-4f88-9e09-373948da2717 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:26.832282 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c99241e5-4d80-4f88-9e09-373948da2717 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:26.846446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c99241e5-4d80-4f88-9e09-373948da2717 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100456 Aug 07 01:20:26.847507 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 518/1066] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:26 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:27.683511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d12d55cc-03d0-44b2-b111-74695e388e32 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:27.685967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d12d55cc-03d0-44b2-b111-74695e388e32 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:27.693361 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d12d55cc-03d0-44b2-b111-74695e388e32 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:27.694602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d12d55cc-03d0-44b2-b111-74695e388e32 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:27.695439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d12d55cc-03d0-44b2-b111-74695e388e32 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:27.696448 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d12d55cc-03d0-44b2-b111-74695e388e32 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:27.697104 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d12d55cc-03d0-44b2-b111-74695e388e32 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:27.697671 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d12d55cc-03d0-44b2-b111-74695e388e32 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:27.736059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d12d55cc-03d0-44b2-b111-74695e388e32 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057581 Aug 07 01:20:27.736690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 549/1067] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:27 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:27.874512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b263bfb4-d238-458d-a09a-fb85dbb1ba4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:27.875580 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b263bfb4-d238-458d-a09a-fb85dbb1ba4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:27.887422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b263bfb4-d238-458d-a09a-fb85dbb1ba4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:27.888355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b263bfb4-d238-458d-a09a-fb85dbb1ba4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:27.889127 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b263bfb4-d238-458d-a09a-fb85dbb1ba4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:27.890360 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b263bfb4-d238-458d-a09a-fb85dbb1ba4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:27.891199 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b263bfb4-d238-458d-a09a-fb85dbb1ba4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:27.891984 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b263bfb4-d238-458d-a09a-fb85dbb1ba4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:27.988835 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b263bfb4-d238-458d-a09a-fb85dbb1ba4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:27.997913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b263bfb4-d238-458d-a09a-fb85dbb1ba4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.130961 Aug 07 01:20:27.998522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 519/1068] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:27 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:28.390426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:20:28.390698 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:20:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:20:28.757019 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-061aea7b-76c4-4dc9-870c-2304826ba325 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:28.760868 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-061aea7b-76c4-4dc9-870c-2304826ba325 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:28.772704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-061aea7b-76c4-4dc9-870c-2304826ba325 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:28.773574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-061aea7b-76c4-4dc9-870c-2304826ba325 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:28.774341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-061aea7b-76c4-4dc9-870c-2304826ba325 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:28.775604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-061aea7b-76c4-4dc9-870c-2304826ba325 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:28.776480 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-061aea7b-76c4-4dc9-870c-2304826ba325 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:28.777282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-061aea7b-76c4-4dc9-870c-2304826ba325 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:28.836620 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-061aea7b-76c4-4dc9-870c-2304826ba325 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.087521 Aug 07 01:20:28.837592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 550/1069] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:28 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:29.020156 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5235b96a-ce11-4cab-83e7-8f8c7cfd9375 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:29.020976 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5235b96a-ce11-4cab-83e7-8f8c7cfd9375 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:29.028701 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5235b96a-ce11-4cab-83e7-8f8c7cfd9375 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:29.029245 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5235b96a-ce11-4cab-83e7-8f8c7cfd9375 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:29.029741 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5235b96a-ce11-4cab-83e7-8f8c7cfd9375 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:29.030691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5235b96a-ce11-4cab-83e7-8f8c7cfd9375 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:29.031128 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5235b96a-ce11-4cab-83e7-8f8c7cfd9375 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:29.031606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5235b96a-ce11-4cab-83e7-8f8c7cfd9375 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:29.102490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5235b96a-ce11-4cab-83e7-8f8c7cfd9375 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:29.110810 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5235b96a-ce11-4cab-83e7-8f8c7cfd9375 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.094710 Aug 07 01:20:29.111474 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 520/1070] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:29 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:29.856050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-451f8031-8800-4231-899c-3be240d3d41f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:29.858627 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-451f8031-8800-4231-899c-3be240d3d41f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:29.865300 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-451f8031-8800-4231-899c-3be240d3d41f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:29.866365 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-451f8031-8800-4231-899c-3be240d3d41f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:29.866804 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-451f8031-8800-4231-899c-3be240d3d41f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:29.867531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-451f8031-8800-4231-899c-3be240d3d41f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:29.867984 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-451f8031-8800-4231-899c-3be240d3d41f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:29.868661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-451f8031-8800-4231-899c-3be240d3d41f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:29.908320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-451f8031-8800-4231-899c-3be240d3d41f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059957 Aug 07 01:20:29.909040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 551/1071] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:29 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:30.140092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fd246c63-e3ee-458c-b12d-7a9dfafe5f5c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:30.141313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fd246c63-e3ee-458c-b12d-7a9dfafe5f5c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:30.152933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd246c63-e3ee-458c-b12d-7a9dfafe5f5c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:30.153837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd246c63-e3ee-458c-b12d-7a9dfafe5f5c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:30.154658 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd246c63-e3ee-458c-b12d-7a9dfafe5f5c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:30.155976 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd246c63-e3ee-458c-b12d-7a9dfafe5f5c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:30.156869 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd246c63-e3ee-458c-b12d-7a9dfafe5f5c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:30.157726 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd246c63-e3ee-458c-b12d-7a9dfafe5f5c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:30.235880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fd246c63-e3ee-458c-b12d-7a9dfafe5f5c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:30.245276 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fd246c63-e3ee-458c-b12d-7a9dfafe5f5c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.113199 Aug 07 01:20:30.245878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 521/1072] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:30 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:30.920831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6dc955cd-3ee4-4bbe-92e2-4509958a2ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:30.923139 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6dc955cd-3ee4-4bbe-92e2-4509958a2ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:30.930849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6dc955cd-3ee4-4bbe-92e2-4509958a2ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:30.931401 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6dc955cd-3ee4-4bbe-92e2-4509958a2ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:30.931954 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6dc955cd-3ee4-4bbe-92e2-4509958a2ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:30.932897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6dc955cd-3ee4-4bbe-92e2-4509958a2ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:30.933510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6dc955cd-3ee4-4bbe-92e2-4509958a2ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:30.934126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6dc955cd-3ee4-4bbe-92e2-4509958a2ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:30.975149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6dc955cd-3ee4-4bbe-92e2-4509958a2ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058217 Aug 07 01:20:30.975741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 552/1073] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:30 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:31.266669 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-511911b7-3bec-43f3-ab2c-72d9db595203 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:31.267304 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-511911b7-3bec-43f3-ab2c-72d9db595203 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:31.274522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-511911b7-3bec-43f3-ab2c-72d9db595203 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:31.275222 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-511911b7-3bec-43f3-ab2c-72d9db595203 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:31.275886 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-511911b7-3bec-43f3-ab2c-72d9db595203 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:31.276815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-511911b7-3bec-43f3-ab2c-72d9db595203 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:31.277399 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-511911b7-3bec-43f3-ab2c-72d9db595203 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:31.277961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-511911b7-3bec-43f3-ab2c-72d9db595203 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:31.300153 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:20:31.353731 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-511911b7-3bec-43f3-ab2c-72d9db595203 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:31.362962 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-511911b7-3bec-43f3-ab2c-72d9db595203 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100213 Aug 07 01:20:31.363556 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 522/1074] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:31 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:31.988281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cd3295e3-34de-45f1-afb4-d105c33e6ba3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:31.990939 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cd3295e3-34de-45f1-afb4-d105c33e6ba3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:31.993543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=56 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:20:31.999553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd3295e3-34de-45f1-afb4-d105c33e6ba3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:32.000114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd3295e3-34de-45f1-afb4-d105c33e6ba3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:32.000671 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd3295e3-34de-45f1-afb4-d105c33e6ba3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:32.001340 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd3295e3-34de-45f1-afb4-d105c33e6ba3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:32.001861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd3295e3-34de-45f1-afb4-d105c33e6ba3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:32.002392 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd3295e3-34de-45f1-afb4-d105c33e6ba3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:32.042191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cd3295e3-34de-45f1-afb4-d105c33e6ba3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058273 Aug 07 01:20:32.042964 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 553/1075] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:31 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:32.385313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cf83053c-19f7-497c-a1b4-b733945b2684 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:32.386144 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-cf83053c-19f7-497c-a1b4-b733945b2684 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:32.393325 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf83053c-19f7-497c-a1b4-b733945b2684 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:32.393916 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf83053c-19f7-497c-a1b4-b733945b2684 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:32.394532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf83053c-19f7-497c-a1b4-b733945b2684 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:32.395489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf83053c-19f7-497c-a1b4-b733945b2684 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:32.396018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf83053c-19f7-497c-a1b4-b733945b2684 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:32.396566 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf83053c-19f7-497c-a1b4-b733945b2684 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:32.468545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-cf83053c-19f7-497c-a1b4-b733945b2684 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:32.479493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-cf83053c-19f7-497c-a1b4-b733945b2684 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097724 Aug 07 01:20:32.480917 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 523/1076] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:32 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:33.058635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3ae58b31-9f8a-446d-a447-7b80c9e6c789 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:33.062447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3ae58b31-9f8a-446d-a447-7b80c9e6c789 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:33.073800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae58b31-9f8a-446d-a447-7b80c9e6c789 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:33.074491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae58b31-9f8a-446d-a447-7b80c9e6c789 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:33.075536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae58b31-9f8a-446d-a447-7b80c9e6c789 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:33.076825 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae58b31-9f8a-446d-a447-7b80c9e6c789 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:33.077640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae58b31-9f8a-446d-a447-7b80c9e6c789 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:33.078455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae58b31-9f8a-446d-a447-7b80c9e6c789 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:33.115507 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3ae58b31-9f8a-446d-a447-7b80c9e6c789 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063625 Aug 07 01:20:33.116102 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 554/1077] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:33 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:33.502539 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a09be978-a9cd-47c1-bd2e-2bf60fcf3311 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:33.503155 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a09be978-a9cd-47c1-bd2e-2bf60fcf3311 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:33.505505 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=54 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:20:33.512272 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a09be978-a9cd-47c1-bd2e-2bf60fcf3311 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:33.512778 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a09be978-a9cd-47c1-bd2e-2bf60fcf3311 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:33.513215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a09be978-a9cd-47c1-bd2e-2bf60fcf3311 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:33.513941 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a09be978-a9cd-47c1-bd2e-2bf60fcf3311 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:33.514394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a09be978-a9cd-47c1-bd2e-2bf60fcf3311 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:33.514915 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a09be978-a9cd-47c1-bd2e-2bf60fcf3311 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:33.586409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a09be978-a9cd-47c1-bd2e-2bf60fcf3311 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:33.594873 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a09be978-a9cd-47c1-bd2e-2bf60fcf3311 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.096552 Aug 07 01:20:33.595450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 524/1078] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:33 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:34.131015 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-de9619e6-1b71-495a-a833-6c40c5af6e2b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:34.133894 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-de9619e6-1b71-495a-a833-6c40c5af6e2b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:34.141553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-de9619e6-1b71-495a-a833-6c40c5af6e2b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:34.142064 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-de9619e6-1b71-495a-a833-6c40c5af6e2b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:34.142510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-de9619e6-1b71-495a-a833-6c40c5af6e2b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:34.143311 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-de9619e6-1b71-495a-a833-6c40c5af6e2b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:34.143943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-de9619e6-1b71-495a-a833-6c40c5af6e2b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:34.144511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-de9619e6-1b71-495a-a833-6c40c5af6e2b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:34.204164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-de9619e6-1b71-495a-a833-6c40c5af6e2b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.076700 Aug 07 01:20:34.208779 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 555/1079] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:34 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:34.617237 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8398d5b3-6c43-4e2c-9ef9-b37900c48ed2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:34.617960 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8398d5b3-6c43-4e2c-9ef9-b37900c48ed2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:34.625716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8398d5b3-6c43-4e2c-9ef9-b37900c48ed2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:34.626534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8398d5b3-6c43-4e2c-9ef9-b37900c48ed2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:34.627239 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8398d5b3-6c43-4e2c-9ef9-b37900c48ed2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:34.628453 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8398d5b3-6c43-4e2c-9ef9-b37900c48ed2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:34.629190 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8398d5b3-6c43-4e2c-9ef9-b37900c48ed2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:34.629870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8398d5b3-6c43-4e2c-9ef9-b37900c48ed2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:34.719914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8398d5b3-6c43-4e2c-9ef9-b37900c48ed2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:34.728460 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8398d5b3-6c43-4e2c-9ef9-b37900c48ed2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.116078 Aug 07 01:20:34.729221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 525/1080] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:34 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:35.218350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c0f07f24-4bba-4db4-9c35-37466edc74da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:35.220594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c0f07f24-4bba-4db4-9c35-37466edc74da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:35.227421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c0f07f24-4bba-4db4-9c35-37466edc74da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:35.227915 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c0f07f24-4bba-4db4-9c35-37466edc74da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:35.228396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c0f07f24-4bba-4db4-9c35-37466edc74da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:35.229124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c0f07f24-4bba-4db4-9c35-37466edc74da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:35.229582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c0f07f24-4bba-4db4-9c35-37466edc74da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:35.230017 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c0f07f24-4bba-4db4-9c35-37466edc74da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:35.263945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c0f07f24-4bba-4db4-9c35-37466edc74da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049374 Aug 07 01:20:35.264530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 556/1081] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:35 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:35.749938 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7a29b07e-f948-4fc0-93e3-c87f9ed1bcb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:35.750600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7a29b07e-f948-4fc0-93e3-c87f9ed1bcb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:35.760904 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a29b07e-f948-4fc0-93e3-c87f9ed1bcb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:35.761797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a29b07e-f948-4fc0-93e3-c87f9ed1bcb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:35.762599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a29b07e-f948-4fc0-93e3-c87f9ed1bcb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:35.763874 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a29b07e-f948-4fc0-93e3-c87f9ed1bcb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:35.764725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a29b07e-f948-4fc0-93e3-c87f9ed1bcb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:35.765475 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a29b07e-f948-4fc0-93e3-c87f9ed1bcb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:35.846904 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7a29b07e-f948-4fc0-93e3-c87f9ed1bcb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:35.856834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7a29b07e-f948-4fc0-93e3-c87f9ed1bcb7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.110765 Aug 07 01:20:35.857591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 526/1082] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:35 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:36.278288 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2d52c6dc-f006-4333-a62d-c24e538e2b06 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:36.280644 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2d52c6dc-f006-4333-a62d-c24e538e2b06 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:36.288849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d52c6dc-f006-4333-a62d-c24e538e2b06 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:36.289491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d52c6dc-f006-4333-a62d-c24e538e2b06 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:36.290046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d52c6dc-f006-4333-a62d-c24e538e2b06 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:36.290964 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d52c6dc-f006-4333-a62d-c24e538e2b06 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:36.291522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d52c6dc-f006-4333-a62d-c24e538e2b06 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:36.292050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d52c6dc-f006-4333-a62d-c24e538e2b06 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:36.329305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2d52c6dc-f006-4333-a62d-c24e538e2b06 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055804 Aug 07 01:20:36.329931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 557/1083] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:36 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:36.879728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b1a4b444-27c7-4bfa-a70d-c04d82aeaa9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:36.881158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b1a4b444-27c7-4bfa-a70d-c04d82aeaa9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:36.888645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b1a4b444-27c7-4bfa-a70d-c04d82aeaa9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:36.889277 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b1a4b444-27c7-4bfa-a70d-c04d82aeaa9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:36.889807 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b1a4b444-27c7-4bfa-a70d-c04d82aeaa9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:36.890714 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b1a4b444-27c7-4bfa-a70d-c04d82aeaa9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:36.891325 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b1a4b444-27c7-4bfa-a70d-c04d82aeaa9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:36.891826 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b1a4b444-27c7-4bfa-a70d-c04d82aeaa9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:36.971512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b1a4b444-27c7-4bfa-a70d-c04d82aeaa9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:36.980322 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b1a4b444-27c7-4bfa-a70d-c04d82aeaa9e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.104966 Aug 07 01:20:36.981043 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 527/1084] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:36 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:37.344404 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-17152d60-dab1-485e-bee5-62c14b1928c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:37.346660 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-17152d60-dab1-485e-bee5-62c14b1928c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:37.354491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17152d60-dab1-485e-bee5-62c14b1928c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:37.354985 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17152d60-dab1-485e-bee5-62c14b1928c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:37.355430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17152d60-dab1-485e-bee5-62c14b1928c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:37.356152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17152d60-dab1-485e-bee5-62c14b1928c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:37.356639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17152d60-dab1-485e-bee5-62c14b1928c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:37.357070 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17152d60-dab1-485e-bee5-62c14b1928c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:37.391584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-17152d60-dab1-485e-bee5-62c14b1928c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051361 Aug 07 01:20:37.392175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 558/1085] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:37 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:38.003680 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f29b6bdd-2203-47ea-8c3b-0950667ce821 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:38.004216 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f29b6bdd-2203-47ea-8c3b-0950667ce821 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:38.012880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f29b6bdd-2203-47ea-8c3b-0950667ce821 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:38.013490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f29b6bdd-2203-47ea-8c3b-0950667ce821 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:38.013967 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f29b6bdd-2203-47ea-8c3b-0950667ce821 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:38.014770 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f29b6bdd-2203-47ea-8c3b-0950667ce821 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:38.015284 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f29b6bdd-2203-47ea-8c3b-0950667ce821 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:38.015752 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f29b6bdd-2203-47ea-8c3b-0950667ce821 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:38.092268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f29b6bdd-2203-47ea-8c3b-0950667ce821 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:38.101734 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f29b6bdd-2203-47ea-8c3b-0950667ce821 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.102498 Aug 07 01:20:38.102433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 528/1086] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:37 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:38.404728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b62231e5-4438-47dc-9679-177dbfc73589 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:38.407293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b62231e5-4438-47dc-9679-177dbfc73589 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:38.415541 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b62231e5-4438-47dc-9679-177dbfc73589 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:38.416042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b62231e5-4438-47dc-9679-177dbfc73589 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:38.416589 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b62231e5-4438-47dc-9679-177dbfc73589 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:38.417430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b62231e5-4438-47dc-9679-177dbfc73589 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:38.418123 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b62231e5-4438-47dc-9679-177dbfc73589 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:38.418661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b62231e5-4438-47dc-9679-177dbfc73589 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:38.453584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b62231e5-4438-47dc-9679-177dbfc73589 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052913 Aug 07 01:20:38.454191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 559/1087] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:38 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:39.136620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ea1e4ed9-c1ff-42ea-820b-d5c444d2d359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:39.137447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ea1e4ed9-c1ff-42ea-820b-d5c444d2d359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:39.144307 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ea1e4ed9-c1ff-42ea-820b-d5c444d2d359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:39.144811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ea1e4ed9-c1ff-42ea-820b-d5c444d2d359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:39.145251 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ea1e4ed9-c1ff-42ea-820b-d5c444d2d359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:39.146056 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ea1e4ed9-c1ff-42ea-820b-d5c444d2d359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:39.147075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ea1e4ed9-c1ff-42ea-820b-d5c444d2d359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:39.147507 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ea1e4ed9-c1ff-42ea-820b-d5c444d2d359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:39.217773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ea1e4ed9-c1ff-42ea-820b-d5c444d2d359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:39.226678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ea1e4ed9-c1ff-42ea-820b-d5c444d2d359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.094024 Aug 07 01:20:39.227431 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 529/1088] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:39 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:39.466034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-08ae3bf0-4906-4c64-99f5-d68a99f5859c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:39.468485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-08ae3bf0-4906-4c64-99f5-d68a99f5859c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:39.475309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-08ae3bf0-4906-4c64-99f5-d68a99f5859c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:39.475802 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-08ae3bf0-4906-4c64-99f5-d68a99f5859c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:39.476278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-08ae3bf0-4906-4c64-99f5-d68a99f5859c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:39.477015 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-08ae3bf0-4906-4c64-99f5-d68a99f5859c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:39.477482 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-08ae3bf0-4906-4c64-99f5-d68a99f5859c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:39.477909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-08ae3bf0-4906-4c64-99f5-d68a99f5859c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:39.513569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-08ae3bf0-4906-4c64-99f5-d68a99f5859c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051828 Aug 07 01:20:39.514157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 560/1089] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:39 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:40.248581 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-aba9f028-68e7-4d7b-98ec-456bea74ad72 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:40.249319 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-aba9f028-68e7-4d7b-98ec-456bea74ad72 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:40.258012 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aba9f028-68e7-4d7b-98ec-456bea74ad72 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:40.258588 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aba9f028-68e7-4d7b-98ec-456bea74ad72 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:40.259082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aba9f028-68e7-4d7b-98ec-456bea74ad72 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:40.259941 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aba9f028-68e7-4d7b-98ec-456bea74ad72 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:40.260495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aba9f028-68e7-4d7b-98ec-456bea74ad72 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:40.261053 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aba9f028-68e7-4d7b-98ec-456bea74ad72 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:40.337103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-aba9f028-68e7-4d7b-98ec-456bea74ad72 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:40.346509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-aba9f028-68e7-4d7b-98ec-456bea74ad72 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.101897 Aug 07 01:20:40.347173 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 530/1090] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:40 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:40.526580 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8ade666b-3f3b-479b-9c8c-b66d1da2952b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:40.529190 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8ade666b-3f3b-479b-9c8c-b66d1da2952b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:40.536386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8ade666b-3f3b-479b-9c8c-b66d1da2952b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:40.536910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8ade666b-3f3b-479b-9c8c-b66d1da2952b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:40.537394 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8ade666b-3f3b-479b-9c8c-b66d1da2952b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:40.538412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8ade666b-3f3b-479b-9c8c-b66d1da2952b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:40.538798 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8ade666b-3f3b-479b-9c8c-b66d1da2952b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:40.539263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8ade666b-3f3b-479b-9c8c-b66d1da2952b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:40.574588 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8ade666b-3f3b-479b-9c8c-b66d1da2952b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052036 Aug 07 01:20:40.575515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 561/1091] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:40 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:41.379908 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-043d2494-7d68-4f2f-94bd-03c4d64d6783 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:41.380724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-043d2494-7d68-4f2f-94bd-03c4d64d6783 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:41.387951 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-043d2494-7d68-4f2f-94bd-03c4d64d6783 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:41.388487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-043d2494-7d68-4f2f-94bd-03c4d64d6783 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:41.388952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-043d2494-7d68-4f2f-94bd-03c4d64d6783 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:41.389777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-043d2494-7d68-4f2f-94bd-03c4d64d6783 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:41.390247 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-043d2494-7d68-4f2f-94bd-03c4d64d6783 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:41.390677 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-043d2494-7d68-4f2f-94bd-03c4d64d6783 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:41.481968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-043d2494-7d68-4f2f-94bd-03c4d64d6783 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:41.497773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-043d2494-7d68-4f2f-94bd-03c4d64d6783 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.121648 Aug 07 01:20:41.498865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 531/1092] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:41 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:41.593246 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-28e101ee-c5a0-45b0-8a4b-b3c3bf7b041f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:41.597485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-28e101ee-c5a0-45b0-8a4b-b3c3bf7b041f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:41.610829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-28e101ee-c5a0-45b0-8a4b-b3c3bf7b041f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:41.611324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-28e101ee-c5a0-45b0-8a4b-b3c3bf7b041f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:41.612095 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-28e101ee-c5a0-45b0-8a4b-b3c3bf7b041f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:41.613712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-28e101ee-c5a0-45b0-8a4b-b3c3bf7b041f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:41.614211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-28e101ee-c5a0-45b0-8a4b-b3c3bf7b041f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:41.614664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-28e101ee-c5a0-45b0-8a4b-b3c3bf7b041f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:41.676821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-28e101ee-c5a0-45b0-8a4b-b3c3bf7b041f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.091931 Aug 07 01:20:41.678560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 562/1093] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:41 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:42.522025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1ce57899-4890-4564-bd1f-115da4d567b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:42.522938 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1ce57899-4890-4564-bd1f-115da4d567b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:42.531581 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ce57899-4890-4564-bd1f-115da4d567b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:42.532213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ce57899-4890-4564-bd1f-115da4d567b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:42.532781 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ce57899-4890-4564-bd1f-115da4d567b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:42.533632 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ce57899-4890-4564-bd1f-115da4d567b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:42.534095 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ce57899-4890-4564-bd1f-115da4d567b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:42.534877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ce57899-4890-4564-bd1f-115da4d567b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:42.627522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1ce57899-4890-4564-bd1f-115da4d567b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:42.635496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1ce57899-4890-4564-bd1f-115da4d567b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.117645 Aug 07 01:20:42.636083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 532/1094] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:42 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:42.694689 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e6043183-6f69-431f-a00c-e0f8d1029f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:42.697429 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e6043183-6f69-431f-a00c-e0f8d1029f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:42.704559 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6043183-6f69-431f-a00c-e0f8d1029f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:42.705081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6043183-6f69-431f-a00c-e0f8d1029f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:42.705509 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6043183-6f69-431f-a00c-e0f8d1029f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:42.706240 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6043183-6f69-431f-a00c-e0f8d1029f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:42.706837 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6043183-6f69-431f-a00c-e0f8d1029f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:42.707362 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6043183-6f69-431f-a00c-e0f8d1029f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:42.737232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:20:42.743530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e6043183-6f69-431f-a00c-e0f8d1029f2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053202 Aug 07 01:20:42.743822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 563/1095] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:42 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:43.658815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-726b81b7-9d0f-48eb-95fc-23039a1f7a82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:43.659561 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-726b81b7-9d0f-48eb-95fc-23039a1f7a82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:43.667668 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-726b81b7-9d0f-48eb-95fc-23039a1f7a82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:43.668499 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-726b81b7-9d0f-48eb-95fc-23039a1f7a82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:43.669077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-726b81b7-9d0f-48eb-95fc-23039a1f7a82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:43.669989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-726b81b7-9d0f-48eb-95fc-23039a1f7a82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:43.670576 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-726b81b7-9d0f-48eb-95fc-23039a1f7a82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:43.671199 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-726b81b7-9d0f-48eb-95fc-23039a1f7a82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:43.742641 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-726b81b7-9d0f-48eb-95fc-23039a1f7a82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:43.755135 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-726b81b7-9d0f-48eb-95fc-23039a1f7a82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100677 Aug 07 01:20:43.756691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 533/1096] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:43 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:43.760091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3a939aa1-2e44-49b6-b307-ac87484d88a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:43.762485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3a939aa1-2e44-49b6-b307-ac87484d88a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:43.769944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a939aa1-2e44-49b6-b307-ac87484d88a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:43.770443 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a939aa1-2e44-49b6-b307-ac87484d88a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:43.770899 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a939aa1-2e44-49b6-b307-ac87484d88a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:43.772196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a939aa1-2e44-49b6-b307-ac87484d88a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:43.772681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a939aa1-2e44-49b6-b307-ac87484d88a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:43.773110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a939aa1-2e44-49b6-b307-ac87484d88a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:43.807856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3a939aa1-2e44-49b6-b307-ac87484d88a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052892 Aug 07 01:20:43.808563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 564/1097] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:43 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:44.781015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-41b0927d-2e3c-4948-9f97-2401833f63f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:44.781908 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-41b0927d-2e3c-4948-9f97-2401833f63f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:44.789474 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41b0927d-2e3c-4948-9f97-2401833f63f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:44.789972 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41b0927d-2e3c-4948-9f97-2401833f63f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:44.790452 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41b0927d-2e3c-4948-9f97-2401833f63f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:44.791185 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41b0927d-2e3c-4948-9f97-2401833f63f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:44.791647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41b0927d-2e3c-4948-9f97-2401833f63f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:44.792103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41b0927d-2e3c-4948-9f97-2401833f63f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:44.822681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b807f019-b380-414f-84b0-65f2a0a41445 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:44.826086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b807f019-b380-414f-84b0-65f2a0a41445 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:44.837929 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b807f019-b380-414f-84b0-65f2a0a41445 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:44.839468 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b807f019-b380-414f-84b0-65f2a0a41445 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:44.840589 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b807f019-b380-414f-84b0-65f2a0a41445 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:44.841546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b807f019-b380-414f-84b0-65f2a0a41445 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:44.842261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b807f019-b380-414f-84b0-65f2a0a41445 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:44.842934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b807f019-b380-414f-84b0-65f2a0a41445 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:44.871717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-41b0927d-2e3c-4948-9f97-2401833f63f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:44.890454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-41b0927d-2e3c-4948-9f97-2401833f63f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.112817 Aug 07 01:20:44.892834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 534/1098] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:44 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:44.900487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b807f019-b380-414f-84b0-65f2a0a41445 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.082857 Aug 07 01:20:44.904292 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 565/1099] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:44 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:45.912664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0154b4cd-0877-4a24-9e20-444a56221931 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:45.913584 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0154b4cd-0877-4a24-9e20-444a56221931 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:45.917211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3ad9a37b-078c-479c-86b2-79773c3ebc44 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:45.919528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3ad9a37b-078c-479c-86b2-79773c3ebc44 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:45.922042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0154b4cd-0877-4a24-9e20-444a56221931 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:45.922545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0154b4cd-0877-4a24-9e20-444a56221931 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:45.923076 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0154b4cd-0877-4a24-9e20-444a56221931 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:45.923815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0154b4cd-0877-4a24-9e20-444a56221931 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:45.924304 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0154b4cd-0877-4a24-9e20-444a56221931 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:45.924747 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0154b4cd-0877-4a24-9e20-444a56221931 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:45.926711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ad9a37b-078c-479c-86b2-79773c3ebc44 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:45.927214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ad9a37b-078c-479c-86b2-79773c3ebc44 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:45.927661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ad9a37b-078c-479c-86b2-79773c3ebc44 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:45.928441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ad9a37b-078c-479c-86b2-79773c3ebc44 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:45.928911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ad9a37b-078c-479c-86b2-79773c3ebc44 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:45.929450 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ad9a37b-078c-479c-86b2-79773c3ebc44 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:45.971085 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3ad9a37b-078c-479c-86b2-79773c3ebc44 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057648 Aug 07 01:20:45.972183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 566/1100] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:45 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:46.000931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0154b4cd-0877-4a24-9e20-444a56221931 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:46.009840 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0154b4cd-0877-4a24-9e20-444a56221931 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.102381 Aug 07 01:20:46.010464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 535/1101] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:45 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:46.988554 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7e87b366-e21b-47f0-9905-3e5cbb31f971 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:46.991688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7e87b366-e21b-47f0-9905-3e5cbb31f971 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:46.999093 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e87b366-e21b-47f0-9905-3e5cbb31f971 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:46.999885 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e87b366-e21b-47f0-9905-3e5cbb31f971 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:47.000440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e87b366-e21b-47f0-9905-3e5cbb31f971 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:47.001364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e87b366-e21b-47f0-9905-3e5cbb31f971 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:47.001892 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e87b366-e21b-47f0-9905-3e5cbb31f971 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:47.002465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e87b366-e21b-47f0-9905-3e5cbb31f971 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:47.032700 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a2268ede-cc4e-4d8e-a453-d547db4721bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:47.033586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a2268ede-cc4e-4d8e-a453-d547db4721bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:47.039259 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7e87b366-e21b-47f0-9905-3e5cbb31f971 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054929 Aug 07 01:20:47.040500 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 567/1102] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:46 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:47.041441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2268ede-cc4e-4d8e-a453-d547db4721bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:47.041933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2268ede-cc4e-4d8e-a453-d547db4721bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:47.042368 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2268ede-cc4e-4d8e-a453-d547db4721bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:47.043174 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2268ede-cc4e-4d8e-a453-d547db4721bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:47.043832 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2268ede-cc4e-4d8e-a453-d547db4721bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:47.044465 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2268ede-cc4e-4d8e-a453-d547db4721bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:47.126654 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a2268ede-cc4e-4d8e-a453-d547db4721bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:47.135396 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a2268ede-cc4e-4d8e-a453-d547db4721bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.107143 Aug 07 01:20:47.136329 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 536/1103] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:47 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:48.053948 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a4d17ef7-52dc-40aa-8c6e-dd761565dfb8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:48.056207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a4d17ef7-52dc-40aa-8c6e-dd761565dfb8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:48.063897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4d17ef7-52dc-40aa-8c6e-dd761565dfb8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:48.064569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4d17ef7-52dc-40aa-8c6e-dd761565dfb8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:48.065125 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4d17ef7-52dc-40aa-8c6e-dd761565dfb8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:48.065979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4d17ef7-52dc-40aa-8c6e-dd761565dfb8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:48.066540 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4d17ef7-52dc-40aa-8c6e-dd761565dfb8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:48.067078 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4d17ef7-52dc-40aa-8c6e-dd761565dfb8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:48.106086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a4d17ef7-52dc-40aa-8c6e-dd761565dfb8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056168 Aug 07 01:20:48.106722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 568/1104] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:48 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:48.157029 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0b161718-a33d-4f18-8364-8be90466228f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:48.157772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0b161718-a33d-4f18-8364-8be90466228f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:48.165841 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b161718-a33d-4f18-8364-8be90466228f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:48.166333 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b161718-a33d-4f18-8364-8be90466228f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:48.166815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b161718-a33d-4f18-8364-8be90466228f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:48.167631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b161718-a33d-4f18-8364-8be90466228f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:48.168211 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b161718-a33d-4f18-8364-8be90466228f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:48.168673 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b161718-a33d-4f18-8364-8be90466228f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:48.238702 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0b161718-a33d-4f18-8364-8be90466228f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:48.247481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0b161718-a33d-4f18-8364-8be90466228f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.094422 Aug 07 01:20:48.248447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 537/1105] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:48 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:49.120716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b1b63ee5-9bfc-494a-a508-d1430458aa9d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:49.122960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b1b63ee5-9bfc-494a-a508-d1430458aa9d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:49.130093 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b1b63ee5-9bfc-494a-a508-d1430458aa9d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:49.130589 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b1b63ee5-9bfc-494a-a508-d1430458aa9d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:49.131121 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b1b63ee5-9bfc-494a-a508-d1430458aa9d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:49.131942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b1b63ee5-9bfc-494a-a508-d1430458aa9d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:49.132536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b1b63ee5-9bfc-494a-a508-d1430458aa9d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:49.132967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b1b63ee5-9bfc-494a-a508-d1430458aa9d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:49.168583 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b1b63ee5-9bfc-494a-a508-d1430458aa9d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052336 Aug 07 01:20:49.169173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 569/1106] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:49 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:49.270436 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bc86f3d0-4cb1-48fe-beb3-f9266ab23833 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:49.271196 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bc86f3d0-4cb1-48fe-beb3-f9266ab23833 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:49.279365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc86f3d0-4cb1-48fe-beb3-f9266ab23833 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:49.280144 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc86f3d0-4cb1-48fe-beb3-f9266ab23833 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:49.282992 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc86f3d0-4cb1-48fe-beb3-f9266ab23833 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:49.283804 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc86f3d0-4cb1-48fe-beb3-f9266ab23833 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:49.284720 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc86f3d0-4cb1-48fe-beb3-f9266ab23833 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:49.285599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc86f3d0-4cb1-48fe-beb3-f9266ab23833 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:49.356486 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bc86f3d0-4cb1-48fe-beb3-f9266ab23833 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:49.365946 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bc86f3d0-4cb1-48fe-beb3-f9266ab23833 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.099658 Aug 07 01:20:49.366556 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 538/1107] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:49 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:50.182433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fa3cc590-b175-4512-88d8-7812e9b68246 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:50.184899 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fa3cc590-b175-4512-88d8-7812e9b68246 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:50.192432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fa3cc590-b175-4512-88d8-7812e9b68246 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:50.193534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fa3cc590-b175-4512-88d8-7812e9b68246 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:50.193978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fa3cc590-b175-4512-88d8-7812e9b68246 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:50.194709 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fa3cc590-b175-4512-88d8-7812e9b68246 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:50.195167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fa3cc590-b175-4512-88d8-7812e9b68246 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:50.195594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fa3cc590-b175-4512-88d8-7812e9b68246 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:50.241893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fa3cc590-b175-4512-88d8-7812e9b68246 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063498 Aug 07 01:20:50.242606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 570/1108] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:50 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:50.387127 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-643e35f9-4763-4fd7-b89e-4100b67bb4ee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:50.388159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-643e35f9-4763-4fd7-b89e-4100b67bb4ee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:50.396389 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-643e35f9-4763-4fd7-b89e-4100b67bb4ee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:50.397033 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-643e35f9-4763-4fd7-b89e-4100b67bb4ee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:50.397579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-643e35f9-4763-4fd7-b89e-4100b67bb4ee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:50.398428 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-643e35f9-4763-4fd7-b89e-4100b67bb4ee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:50.399002 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-643e35f9-4763-4fd7-b89e-4100b67bb4ee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:50.399638 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-643e35f9-4763-4fd7-b89e-4100b67bb4ee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:50.404777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:20:50.486667 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-643e35f9-4763-4fd7-b89e-4100b67bb4ee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:50.495775 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-643e35f9-4763-4fd7-b89e-4100b67bb4ee tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.112504 Aug 07 01:20:50.496507 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 539/1109] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:50 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:51.258908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8473cb93-0cea-4b59-bb38-4c823c267602 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:51.261296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8473cb93-0cea-4b59-bb38-4c823c267602 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:51.268437 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8473cb93-0cea-4b59-bb38-4c823c267602 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:51.268960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8473cb93-0cea-4b59-bb38-4c823c267602 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:51.269406 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8473cb93-0cea-4b59-bb38-4c823c267602 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:51.270279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8473cb93-0cea-4b59-bb38-4c823c267602 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:51.271316 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8473cb93-0cea-4b59-bb38-4c823c267602 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:51.271750 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8473cb93-0cea-4b59-bb38-4c823c267602 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:51.310568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8473cb93-0cea-4b59-bb38-4c823c267602 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056746 Aug 07 01:20:51.311292 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 571/1110] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:51 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:51.519415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-03a3f45f-0557-4119-b5da-92958ac9d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:51.520177 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-03a3f45f-0557-4119-b5da-92958ac9d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:51.528229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-03a3f45f-0557-4119-b5da-92958ac9d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:51.528774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-03a3f45f-0557-4119-b5da-92958ac9d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:51.529214 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-03a3f45f-0557-4119-b5da-92958ac9d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:51.529953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-03a3f45f-0557-4119-b5da-92958ac9d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:51.530521 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-03a3f45f-0557-4119-b5da-92958ac9d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:51.530952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-03a3f45f-0557-4119-b5da-92958ac9d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:51.619690 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-03a3f45f-0557-4119-b5da-92958ac9d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:51.630045 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-03a3f45f-0557-4119-b5da-92958ac9d391 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.114575 Aug 07 01:20:51.630722 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 540/1111] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:51 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:52.324339 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-65a4e94a-32b6-4d6f-b3a3-104c9a842f4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:52.326855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-65a4e94a-32b6-4d6f-b3a3-104c9a842f4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:52.334501 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-65a4e94a-32b6-4d6f-b3a3-104c9a842f4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:52.335004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-65a4e94a-32b6-4d6f-b3a3-104c9a842f4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:52.335449 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-65a4e94a-32b6-4d6f-b3a3-104c9a842f4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:52.336181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-65a4e94a-32b6-4d6f-b3a3-104c9a842f4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:52.336694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-65a4e94a-32b6-4d6f-b3a3-104c9a842f4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:52.337126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-65a4e94a-32b6-4d6f-b3a3-104c9a842f4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:52.373372 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-65a4e94a-32b6-4d6f-b3a3-104c9a842f4d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053163 Aug 07 01:20:52.374014 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 572/1112] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:52 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:52.657446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-630123dd-0c4b-4175-b440-4d7381070623 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:52.658483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-630123dd-0c4b-4175-b440-4d7381070623 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:52.670700 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-630123dd-0c4b-4175-b440-4d7381070623 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:52.671555 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-630123dd-0c4b-4175-b440-4d7381070623 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:52.672353 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-630123dd-0c4b-4175-b440-4d7381070623 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:52.673629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-630123dd-0c4b-4175-b440-4d7381070623 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:52.674445 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-630123dd-0c4b-4175-b440-4d7381070623 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:52.675202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-630123dd-0c4b-4175-b440-4d7381070623 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:52.771728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-630123dd-0c4b-4175-b440-4d7381070623 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:52.781591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-630123dd-0c4b-4175-b440-4d7381070623 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.131336 Aug 07 01:20:52.782352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 541/1113] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:52 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:53.387354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-65a1930b-fb20-4adf-b02a-933f96905175 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:53.389826 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-65a1930b-fb20-4adf-b02a-933f96905175 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:53.397640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-65a1930b-fb20-4adf-b02a-933f96905175 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:53.398138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-65a1930b-fb20-4adf-b02a-933f96905175 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:53.398582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-65a1930b-fb20-4adf-b02a-933f96905175 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:53.399305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-65a1930b-fb20-4adf-b02a-933f96905175 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:53.399779 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-65a1930b-fb20-4adf-b02a-933f96905175 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:53.400197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-65a1930b-fb20-4adf-b02a-933f96905175 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:53.435945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-65a1930b-fb20-4adf-b02a-933f96905175 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052743 Aug 07 01:20:53.436686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 573/1114] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:53 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:53.806346 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-16289b0a-ca34-439d-8df6-fa434820c39c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:53.807215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-16289b0a-ca34-439d-8df6-fa434820c39c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:53.814396 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16289b0a-ca34-439d-8df6-fa434820c39c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:53.814944 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16289b0a-ca34-439d-8df6-fa434820c39c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:53.815442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16289b0a-ca34-439d-8df6-fa434820c39c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:53.816257 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16289b0a-ca34-439d-8df6-fa434820c39c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:53.816960 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16289b0a-ca34-439d-8df6-fa434820c39c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:53.817942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16289b0a-ca34-439d-8df6-fa434820c39c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:53.899324 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-16289b0a-ca34-439d-8df6-fa434820c39c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:53.913583 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-16289b0a-ca34-439d-8df6-fa434820c39c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.110997 Aug 07 01:20:53.914629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 542/1115] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:53 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:20:54.453262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a807b06e-0ee8-46dd-9ef0-de2c29edc55c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:54.455421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a807b06e-0ee8-46dd-9ef0-de2c29edc55c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:54.463347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a807b06e-0ee8-46dd-9ef0-de2c29edc55c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:54.463906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a807b06e-0ee8-46dd-9ef0-de2c29edc55c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:54.464436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a807b06e-0ee8-46dd-9ef0-de2c29edc55c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:54.465212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a807b06e-0ee8-46dd-9ef0-de2c29edc55c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:54.465718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a807b06e-0ee8-46dd-9ef0-de2c29edc55c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:54.466203 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a807b06e-0ee8-46dd-9ef0-de2c29edc55c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:54.500617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a807b06e-0ee8-46dd-9ef0-de2c29edc55c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052334 Aug 07 01:20:54.501263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 574/1116] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:54 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:54.950481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4b159af0-b339-4243-9bfa-12a437f3d6d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:54.951528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4b159af0-b339-4243-9bfa-12a437f3d6d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:54.959217 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b159af0-b339-4243-9bfa-12a437f3d6d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:54.959864 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b159af0-b339-4243-9bfa-12a437f3d6d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:54.960473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b159af0-b339-4243-9bfa-12a437f3d6d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:54.961303 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b159af0-b339-4243-9bfa-12a437f3d6d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:54.962068 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b159af0-b339-4243-9bfa-12a437f3d6d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:54.962734 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4b159af0-b339-4243-9bfa-12a437f3d6d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:55.033689 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4b159af0-b339-4243-9bfa-12a437f3d6d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:55.043100 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4b159af0-b339-4243-9bfa-12a437f3d6d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097125 Aug 07 01:20:55.043698 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 543/1117] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:54 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:55.514487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dae9638c-5159-46d3-8f19-c93814524dc6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:55.516829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dae9638c-5159-46d3-8f19-c93814524dc6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:55.524478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dae9638c-5159-46d3-8f19-c93814524dc6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:55.524979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dae9638c-5159-46d3-8f19-c93814524dc6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:55.525428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dae9638c-5159-46d3-8f19-c93814524dc6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:55.526182 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dae9638c-5159-46d3-8f19-c93814524dc6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:55.526691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dae9638c-5159-46d3-8f19-c93814524dc6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:55.527129 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dae9638c-5159-46d3-8f19-c93814524dc6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:55.565086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dae9638c-5159-46d3-8f19-c93814524dc6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054656 Aug 07 01:20:55.565589 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 575/1118] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:55 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:56.075976 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8d07317c-8ec1-4939-b814-0c5925d5c028 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:56.076972 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8d07317c-8ec1-4939-b814-0c5925d5c028 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:56.085049 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d07317c-8ec1-4939-b814-0c5925d5c028 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:56.085582 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d07317c-8ec1-4939-b814-0c5925d5c028 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:56.086058 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d07317c-8ec1-4939-b814-0c5925d5c028 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:56.086799 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d07317c-8ec1-4939-b814-0c5925d5c028 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:56.087266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d07317c-8ec1-4939-b814-0c5925d5c028 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:56.087699 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d07317c-8ec1-4939-b814-0c5925d5c028 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:56.187112 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8d07317c-8ec1-4939-b814-0c5925d5c028 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:56.196978 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8d07317c-8ec1-4939-b814-0c5925d5c028 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.125567 Aug 07 01:20:56.197866 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 544/1119] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:56 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:56.578513 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7e5d83b5-e127-4710-a902-7bc10f4162ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:56.580866 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7e5d83b5-e127-4710-a902-7bc10f4162ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:56.588034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e5d83b5-e127-4710-a902-7bc10f4162ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:56.588562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e5d83b5-e127-4710-a902-7bc10f4162ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:56.589006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e5d83b5-e127-4710-a902-7bc10f4162ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:56.589737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e5d83b5-e127-4710-a902-7bc10f4162ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:56.590223 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e5d83b5-e127-4710-a902-7bc10f4162ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:56.590658 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7e5d83b5-e127-4710-a902-7bc10f4162ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:56.626814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7e5d83b5-e127-4710-a902-7bc10f4162ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052553 Aug 07 01:20:56.627716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 576/1120] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:56 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:20:57.222925 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8698cee6-7c77-4031-9295-dbfc3e03ab8f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:57.224742 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8698cee6-7c77-4031-9295-dbfc3e03ab8f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:57.236416 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8698cee6-7c77-4031-9295-dbfc3e03ab8f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:57.237341 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8698cee6-7c77-4031-9295-dbfc3e03ab8f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:57.238211 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8698cee6-7c77-4031-9295-dbfc3e03ab8f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:57.239664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8698cee6-7c77-4031-9295-dbfc3e03ab8f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:57.240580 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8698cee6-7c77-4031-9295-dbfc3e03ab8f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:57.241418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8698cee6-7c77-4031-9295-dbfc3e03ab8f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:57.317881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8698cee6-7c77-4031-9295-dbfc3e03ab8f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:57.326625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8698cee6-7c77-4031-9295-dbfc3e03ab8f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.111461 Aug 07 01:20:57.327209 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 545/1121] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:57 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:57.639760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3c4bdbea-a1fe-46bb-b56c-2f2f7c5b217f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:57.642052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3c4bdbea-a1fe-46bb-b56c-2f2f7c5b217f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:57.651060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c4bdbea-a1fe-46bb-b56c-2f2f7c5b217f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:57.651566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c4bdbea-a1fe-46bb-b56c-2f2f7c5b217f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:57.652011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c4bdbea-a1fe-46bb-b56c-2f2f7c5b217f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:57.652794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c4bdbea-a1fe-46bb-b56c-2f2f7c5b217f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:57.653256 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c4bdbea-a1fe-46bb-b56c-2f2f7c5b217f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:57.653701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c4bdbea-a1fe-46bb-b56c-2f2f7c5b217f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:57.688555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3c4bdbea-a1fe-46bb-b56c-2f2f7c5b217f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052848 Aug 07 01:20:57.689341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 577/1122] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:57 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:58.349780 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9704a2ce-2916-49e9-b10e-980707e3c7e3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:58.350634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9704a2ce-2916-49e9-b10e-980707e3c7e3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:58.358254 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9704a2ce-2916-49e9-b10e-980707e3c7e3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:58.358830 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9704a2ce-2916-49e9-b10e-980707e3c7e3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:58.359274 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9704a2ce-2916-49e9-b10e-980707e3c7e3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:58.360229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9704a2ce-2916-49e9-b10e-980707e3c7e3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:58.360509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9704a2ce-2916-49e9-b10e-980707e3c7e3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:58.360942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9704a2ce-2916-49e9-b10e-980707e3c7e3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:58.429292 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9704a2ce-2916-49e9-b10e-980707e3c7e3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:58.438263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9704a2ce-2916-49e9-b10e-980707e3c7e3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.093094 Aug 07 01:20:58.438897 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 546/1123] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:58 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:58.704360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-909c3770-21a3-480d-b95a-12555990130f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:58.706872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-909c3770-21a3-480d-b95a-12555990130f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:58.714849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-909c3770-21a3-480d-b95a-12555990130f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:58.715448 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-909c3770-21a3-480d-b95a-12555990130f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:58.715941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-909c3770-21a3-480d-b95a-12555990130f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:58.716796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-909c3770-21a3-480d-b95a-12555990130f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:58.717321 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-909c3770-21a3-480d-b95a-12555990130f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:58.717874 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-909c3770-21a3-480d-b95a-12555990130f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:58.752651 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-909c3770-21a3-480d-b95a-12555990130f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052792 Aug 07 01:20:58.753260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 578/1124] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:58 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:20:59.463490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-34c9c36e-1cbe-49fc-85ee-f41edaa3a546 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:59.465119 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-34c9c36e-1cbe-49fc-85ee-f41edaa3a546 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:59.476934 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34c9c36e-1cbe-49fc-85ee-f41edaa3a546 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:59.477881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34c9c36e-1cbe-49fc-85ee-f41edaa3a546 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:59.478528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34c9c36e-1cbe-49fc-85ee-f41edaa3a546 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:59.479785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34c9c36e-1cbe-49fc-85ee-f41edaa3a546 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:59.480628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34c9c36e-1cbe-49fc-85ee-f41edaa3a546 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:59.481385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34c9c36e-1cbe-49fc-85ee-f41edaa3a546 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:59.570143 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-34c9c36e-1cbe-49fc-85ee-f41edaa3a546 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:20:59.584529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-34c9c36e-1cbe-49fc-85ee-f41edaa3a546 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.127834 Aug 07 01:20:59.585473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 547/1125] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:20:59 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:20:59.768882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-51b6250b-84a6-4c39-ad65-574fa67700ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:20:59.771719 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-51b6250b-84a6-4c39-ad65-574fa67700ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:20:59.780722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51b6250b-84a6-4c39-ad65-574fa67700ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:59.781327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51b6250b-84a6-4c39-ad65-574fa67700ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:59.781809 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51b6250b-84a6-4c39-ad65-574fa67700ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:59.782669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51b6250b-84a6-4c39-ad65-574fa67700ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:20:59.783183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51b6250b-84a6-4c39-ad65-574fa67700ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:20:59.783648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51b6250b-84a6-4c39-ad65-574fa67700ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:20:59.820214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-51b6250b-84a6-4c39-ad65-574fa67700ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055406 Aug 07 01:20:59.820848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 579/1126] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:20:59 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:00.617759 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ca20b4a6-3408-4197-997b-7e209b142bb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:00.618439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ca20b4a6-3408-4197-997b-7e209b142bb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:00.625129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca20b4a6-3408-4197-997b-7e209b142bb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:00.625631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca20b4a6-3408-4197-997b-7e209b142bb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:00.626072 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca20b4a6-3408-4197-997b-7e209b142bb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:00.626811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca20b4a6-3408-4197-997b-7e209b142bb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:00.627351 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca20b4a6-3408-4197-997b-7e209b142bb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:00.627778 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca20b4a6-3408-4197-997b-7e209b142bb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:00.712975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ca20b4a6-3408-4197-997b-7e209b142bb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:00.721929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ca20b4a6-3408-4197-997b-7e209b142bb4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.108988 Aug 07 01:21:00.722535 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 548/1127] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:00 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:00.834065 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8e4c6284-4be4-43c2-96d4-4f00d2074690 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:00.836496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8e4c6284-4be4-43c2-96d4-4f00d2074690 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:00.843505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e4c6284-4be4-43c2-96d4-4f00d2074690 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:00.844003 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e4c6284-4be4-43c2-96d4-4f00d2074690 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:00.844489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e4c6284-4be4-43c2-96d4-4f00d2074690 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:00.845233 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e4c6284-4be4-43c2-96d4-4f00d2074690 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:00.845693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e4c6284-4be4-43c2-96d4-4f00d2074690 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:00.846133 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e4c6284-4be4-43c2-96d4-4f00d2074690 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:00.871703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:21:00.888486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8e4c6284-4be4-43c2-96d4-4f00d2074690 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058865 Aug 07 01:21:00.889113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 580/1128] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:00 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:01.744836 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e1726935-4cdd-4176-b51e-dbacd70d3ec1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:01.745715 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e1726935-4cdd-4176-b51e-dbacd70d3ec1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:01.753068 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e1726935-4cdd-4176-b51e-dbacd70d3ec1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:01.753719 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e1726935-4cdd-4176-b51e-dbacd70d3ec1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:01.754354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e1726935-4cdd-4176-b51e-dbacd70d3ec1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:01.755398 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e1726935-4cdd-4176-b51e-dbacd70d3ec1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:01.755916 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e1726935-4cdd-4176-b51e-dbacd70d3ec1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:01.756449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e1726935-4cdd-4176-b51e-dbacd70d3ec1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:01.826756 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e1726935-4cdd-4176-b51e-dbacd70d3ec1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:01.835771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e1726935-4cdd-4176-b51e-dbacd70d3ec1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.095409 Aug 07 01:21:01.836409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 549/1129] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:01 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:01.909737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-43ee8024-daa4-437f-9cc1-adf936e708dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:01.914068 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-43ee8024-daa4-437f-9cc1-adf936e708dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:01.925489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-43ee8024-daa4-437f-9cc1-adf936e708dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:01.926312 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-43ee8024-daa4-437f-9cc1-adf936e708dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:01.927090 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-43ee8024-daa4-437f-9cc1-adf936e708dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:01.928410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-43ee8024-daa4-437f-9cc1-adf936e708dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:01.929288 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-43ee8024-daa4-437f-9cc1-adf936e708dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:01.930004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-43ee8024-daa4-437f-9cc1-adf936e708dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:01.967381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-43ee8024-daa4-437f-9cc1-adf936e708dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065634 Aug 07 01:21:01.968056 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 581/1130] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:01 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:02.859590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-05d8b7dc-cbb6-49db-8aca-cde81077cde4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:02.860202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-05d8b7dc-cbb6-49db-8aca-cde81077cde4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:02.867628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-05d8b7dc-cbb6-49db-8aca-cde81077cde4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:02.868121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-05d8b7dc-cbb6-49db-8aca-cde81077cde4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:02.868585 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-05d8b7dc-cbb6-49db-8aca-cde81077cde4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:02.869300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-05d8b7dc-cbb6-49db-8aca-cde81077cde4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:02.869761 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-05d8b7dc-cbb6-49db-8aca-cde81077cde4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:02.870234 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-05d8b7dc-cbb6-49db-8aca-cde81077cde4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:02.946642 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-05d8b7dc-cbb6-49db-8aca-cde81077cde4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:02.955870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-05d8b7dc-cbb6-49db-8aca-cde81077cde4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.101422 Aug 07 01:21:02.956522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 550/1131] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:02 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:02.983927 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-60d640f2-8ce3-4e01-9a5b-4a0de2839471 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:02.987123 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-60d640f2-8ce3-4e01-9a5b-4a0de2839471 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:02.990155 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=58 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:21:02.996751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60d640f2-8ce3-4e01-9a5b-4a0de2839471 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:02.997402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60d640f2-8ce3-4e01-9a5b-4a0de2839471 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:02.997965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60d640f2-8ce3-4e01-9a5b-4a0de2839471 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:02.998796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60d640f2-8ce3-4e01-9a5b-4a0de2839471 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:02.999348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60d640f2-8ce3-4e01-9a5b-4a0de2839471 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:02.999886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60d640f2-8ce3-4e01-9a5b-4a0de2839471 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:03.036626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-60d640f2-8ce3-4e01-9a5b-4a0de2839471 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060471 Aug 07 01:21:03.037229 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 582/1132] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:02 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:03.978493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-152af93a-5fd6-4737-9163-e47232e92595 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:03.979369 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-152af93a-5fd6-4737-9163-e47232e92595 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:03.982285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=54 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:21:03.988940 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-152af93a-5fd6-4737-9163-e47232e92595 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:03.989547 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-152af93a-5fd6-4737-9163-e47232e92595 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:03.990003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-152af93a-5fd6-4737-9163-e47232e92595 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:03.990747 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-152af93a-5fd6-4737-9163-e47232e92595 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:03.991219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-152af93a-5fd6-4737-9163-e47232e92595 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:03.991732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-152af93a-5fd6-4737-9163-e47232e92595 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:04.054666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-24c540c7-1edc-4528-96e4-87ae1e89b071 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:04.058674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-24c540c7-1edc-4528-96e4-87ae1e89b071 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:04.069562 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-152af93a-5fd6-4737-9163-e47232e92595 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:04.070082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-24c540c7-1edc-4528-96e4-87ae1e89b071 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:04.070717 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-24c540c7-1edc-4528-96e4-87ae1e89b071 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:04.071262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-24c540c7-1edc-4528-96e4-87ae1e89b071 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:04.072140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-24c540c7-1edc-4528-96e4-87ae1e89b071 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:04.072754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-24c540c7-1edc-4528-96e4-87ae1e89b071 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:04.073265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-24c540c7-1edc-4528-96e4-87ae1e89b071 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:04.081089 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-152af93a-5fd6-4737-9163-e47232e92595 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.106347 Aug 07 01:21:04.082049 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 551/1133] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:03 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:04.117172 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-24c540c7-1edc-4528-96e4-87ae1e89b071 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.070869 Aug 07 01:21:04.118082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 583/1134] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:04 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:05.109647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-34714ac0-c6bd-474a-bbe5-84b02bd30fd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:05.110419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-34714ac0-c6bd-474a-bbe5-84b02bd30fd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:05.118721 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34714ac0-c6bd-474a-bbe5-84b02bd30fd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:05.119327 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34714ac0-c6bd-474a-bbe5-84b02bd30fd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:05.119800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34714ac0-c6bd-474a-bbe5-84b02bd30fd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:05.120837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34714ac0-c6bd-474a-bbe5-84b02bd30fd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:05.121339 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34714ac0-c6bd-474a-bbe5-84b02bd30fd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:05.121899 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-34714ac0-c6bd-474a-bbe5-84b02bd30fd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:05.135410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f84b34b8-92a1-49f5-9668-af10be1d076f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:05.138251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f84b34b8-92a1-49f5-9668-af10be1d076f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:05.146626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f84b34b8-92a1-49f5-9668-af10be1d076f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:05.147612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f84b34b8-92a1-49f5-9668-af10be1d076f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:05.148202 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f84b34b8-92a1-49f5-9668-af10be1d076f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:05.149371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f84b34b8-92a1-49f5-9668-af10be1d076f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:05.150280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f84b34b8-92a1-49f5-9668-af10be1d076f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:05.150907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f84b34b8-92a1-49f5-9668-af10be1d076f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:05.197329 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f84b34b8-92a1-49f5-9668-af10be1d076f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066887 Aug 07 01:21:05.198486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 584/1135] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:05 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:05.211007 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-34714ac0-c6bd-474a-bbe5-84b02bd30fd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:05.219211 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-34714ac0-c6bd-474a-bbe5-84b02bd30fd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.113395 Aug 07 01:21:05.219885 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 552/1136] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:05 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:06.211200 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a3fd6731-7558-457a-bfb4-a1bd798c1b9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:06.214082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a3fd6731-7558-457a-bfb4-a1bd798c1b9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:06.221074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a3fd6731-7558-457a-bfb4-a1bd798c1b9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:06.221575 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a3fd6731-7558-457a-bfb4-a1bd798c1b9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:06.222020 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a3fd6731-7558-457a-bfb4-a1bd798c1b9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:06.222745 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a3fd6731-7558-457a-bfb4-a1bd798c1b9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:06.223254 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a3fd6731-7558-457a-bfb4-a1bd798c1b9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:06.223679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a3fd6731-7558-457a-bfb4-a1bd798c1b9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:06.241771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2ba54259-ac48-4b91-a2bf-8245af4ad206 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:06.242500 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2ba54259-ac48-4b91-a2bf-8245af4ad206 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:06.250857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2ba54259-ac48-4b91-a2bf-8245af4ad206 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:06.251489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2ba54259-ac48-4b91-a2bf-8245af4ad206 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:06.251935 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2ba54259-ac48-4b91-a2bf-8245af4ad206 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:06.255268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2ba54259-ac48-4b91-a2bf-8245af4ad206 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:06.255894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2ba54259-ac48-4b91-a2bf-8245af4ad206 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:06.256380 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2ba54259-ac48-4b91-a2bf-8245af4ad206 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:06.261324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a3fd6731-7558-457a-bfb4-a1bd798c1b9f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054765 Aug 07 01:21:06.262294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 585/1137] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:06 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:06.346833 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2ba54259-ac48-4b91-a2bf-8245af4ad206 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:06.355498 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2ba54259-ac48-4b91-a2bf-8245af4ad206 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.117710 Aug 07 01:21:06.356098 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 553/1138] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:06 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:07.277659 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-82380503-ee17-4d75-8582-7939f1f13612 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:07.280112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-82380503-ee17-4d75-8582-7939f1f13612 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:07.289279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-82380503-ee17-4d75-8582-7939f1f13612 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:07.289930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-82380503-ee17-4d75-8582-7939f1f13612 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:07.290495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-82380503-ee17-4d75-8582-7939f1f13612 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:07.291357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-82380503-ee17-4d75-8582-7939f1f13612 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:07.291932 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-82380503-ee17-4d75-8582-7939f1f13612 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:07.292519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-82380503-ee17-4d75-8582-7939f1f13612 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:07.329488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-82380503-ee17-4d75-8582-7939f1f13612 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056478 Aug 07 01:21:07.330086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 586/1139] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:07 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:07.377511 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b35c406a-de3e-4dcc-9928-e730fc4ead90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:07.378368 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b35c406a-de3e-4dcc-9928-e730fc4ead90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:07.385793 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b35c406a-de3e-4dcc-9928-e730fc4ead90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:07.386327 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b35c406a-de3e-4dcc-9928-e730fc4ead90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:07.386720 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b35c406a-de3e-4dcc-9928-e730fc4ead90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:07.387435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b35c406a-de3e-4dcc-9928-e730fc4ead90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:07.387898 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b35c406a-de3e-4dcc-9928-e730fc4ead90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:07.388360 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b35c406a-de3e-4dcc-9928-e730fc4ead90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:07.458066 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b35c406a-de3e-4dcc-9928-e730fc4ead90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:07.466022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b35c406a-de3e-4dcc-9928-e730fc4ead90 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.092553 Aug 07 01:21:07.466613 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 554/1140] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:07 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:08.342322 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-31379644-8f77-40a5-a833-090a84f96021 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:08.344615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-31379644-8f77-40a5-a833-090a84f96021 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:08.352302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31379644-8f77-40a5-a833-090a84f96021 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:08.352800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31379644-8f77-40a5-a833-090a84f96021 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:08.353240 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31379644-8f77-40a5-a833-090a84f96021 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:08.353962 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31379644-8f77-40a5-a833-090a84f96021 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:08.354431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31379644-8f77-40a5-a833-090a84f96021 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:08.354861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31379644-8f77-40a5-a833-090a84f96021 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:08.391619 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-31379644-8f77-40a5-a833-090a84f96021 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053460 Aug 07 01:21:08.392223 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 587/1141] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:08 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:08.487864 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ae404572-d8ce-4124-9c36-4ff839200bc0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:08.489495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ae404572-d8ce-4124-9c36-4ff839200bc0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:08.496002 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae404572-d8ce-4124-9c36-4ff839200bc0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:08.496542 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae404572-d8ce-4124-9c36-4ff839200bc0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:08.496989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae404572-d8ce-4124-9c36-4ff839200bc0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:08.497723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae404572-d8ce-4124-9c36-4ff839200bc0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:08.498185 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae404572-d8ce-4124-9c36-4ff839200bc0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:08.498622 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ae404572-d8ce-4124-9c36-4ff839200bc0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:08.567182 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ae404572-d8ce-4124-9c36-4ff839200bc0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:08.571112 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:21:08.576863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ae404572-d8ce-4124-9c36-4ff839200bc0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.092974 Aug 07 01:21:08.577463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 555/1142] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:08 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:09.408903 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f55d40ff-5f69-42b5-a4c1-d36577be1d54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:09.411157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f55d40ff-5f69-42b5-a4c1-d36577be1d54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:09.418041 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f55d40ff-5f69-42b5-a4c1-d36577be1d54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:09.418561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f55d40ff-5f69-42b5-a4c1-d36577be1d54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:09.419224 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f55d40ff-5f69-42b5-a4c1-d36577be1d54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:09.419975 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f55d40ff-5f69-42b5-a4c1-d36577be1d54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:09.420591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f55d40ff-5f69-42b5-a4c1-d36577be1d54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:09.421038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f55d40ff-5f69-42b5-a4c1-d36577be1d54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:09.459725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f55d40ff-5f69-42b5-a4c1-d36577be1d54 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056347 Aug 07 01:21:09.460633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 588/1143] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:09 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:09.605095 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4042e170-e980-4d82-b4ca-ab87e8998ac4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:09.605882 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4042e170-e980-4d82-b4ca-ab87e8998ac4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:09.612879 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4042e170-e980-4d82-b4ca-ab87e8998ac4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:09.613364 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4042e170-e980-4d82-b4ca-ab87e8998ac4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:09.614416 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4042e170-e980-4d82-b4ca-ab87e8998ac4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:09.615174 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4042e170-e980-4d82-b4ca-ab87e8998ac4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:09.615691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4042e170-e980-4d82-b4ca-ab87e8998ac4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:09.616113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4042e170-e980-4d82-b4ca-ab87e8998ac4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:09.690024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4042e170-e980-4d82-b4ca-ab87e8998ac4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:09.701470 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4042e170-e980-4d82-b4ca-ab87e8998ac4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.104694 Aug 07 01:21:09.703303 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 556/1144] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:09 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:10.480153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3f5ec0c0-ec7a-4875-9b0e-ee772fbc9901 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:10.484214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3f5ec0c0-ec7a-4875-9b0e-ee772fbc9901 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:10.495491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f5ec0c0-ec7a-4875-9b0e-ee772fbc9901 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:10.495992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f5ec0c0-ec7a-4875-9b0e-ee772fbc9901 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:10.496489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f5ec0c0-ec7a-4875-9b0e-ee772fbc9901 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:10.497243 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f5ec0c0-ec7a-4875-9b0e-ee772fbc9901 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:10.497708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f5ec0c0-ec7a-4875-9b0e-ee772fbc9901 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:10.498183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f5ec0c0-ec7a-4875-9b0e-ee772fbc9901 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:10.534324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3f5ec0c0-ec7a-4875-9b0e-ee772fbc9901 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061758 Aug 07 01:21:10.534841 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 589/1145] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:10 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:10.725833 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bd15c94f-efcc-40b7-8b6a-61f48575208f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:10.726463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bd15c94f-efcc-40b7-8b6a-61f48575208f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:10.733482 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bd15c94f-efcc-40b7-8b6a-61f48575208f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:10.734072 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bd15c94f-efcc-40b7-8b6a-61f48575208f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:10.734578 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bd15c94f-efcc-40b7-8b6a-61f48575208f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:10.735376 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bd15c94f-efcc-40b7-8b6a-61f48575208f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:10.735888 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bd15c94f-efcc-40b7-8b6a-61f48575208f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:10.736419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bd15c94f-efcc-40b7-8b6a-61f48575208f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:10.807843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bd15c94f-efcc-40b7-8b6a-61f48575208f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:10.815943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bd15c94f-efcc-40b7-8b6a-61f48575208f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.095870 Aug 07 01:21:10.816662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 557/1146] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:10 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:11.548147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-015f5f8a-170c-4f36-b6ee-5821246b2e1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:11.550657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-015f5f8a-170c-4f36-b6ee-5821246b2e1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:11.558646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-015f5f8a-170c-4f36-b6ee-5821246b2e1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:11.559136 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-015f5f8a-170c-4f36-b6ee-5821246b2e1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:11.559575 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-015f5f8a-170c-4f36-b6ee-5821246b2e1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:11.560331 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-015f5f8a-170c-4f36-b6ee-5821246b2e1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:11.560789 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-015f5f8a-170c-4f36-b6ee-5821246b2e1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:11.561216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-015f5f8a-170c-4f36-b6ee-5821246b2e1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:11.594681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-015f5f8a-170c-4f36-b6ee-5821246b2e1f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050588 Aug 07 01:21:11.595274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 590/1147] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:11 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:11.842259 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7285e897-526b-478c-b066-d3f0c1b093d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:11.843302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7285e897-526b-478c-b066-d3f0c1b093d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:11.854788 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7285e897-526b-478c-b066-d3f0c1b093d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:11.855572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7285e897-526b-478c-b066-d3f0c1b093d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:11.856281 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7285e897-526b-478c-b066-d3f0c1b093d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:11.857463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7285e897-526b-478c-b066-d3f0c1b093d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:11.858150 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7285e897-526b-478c-b066-d3f0c1b093d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:11.858809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7285e897-526b-478c-b066-d3f0c1b093d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:11.936001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7285e897-526b-478c-b066-d3f0c1b093d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:11.945567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7285e897-526b-478c-b066-d3f0c1b093d4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.110594 Aug 07 01:21:11.946191 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 558/1148] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:11 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:12.613038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2da647a8-5e79-4eec-b95a-030ab08a87d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:12.616881 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2da647a8-5e79-4eec-b95a-030ab08a87d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:12.627819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2da647a8-5e79-4eec-b95a-030ab08a87d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:12.628728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2da647a8-5e79-4eec-b95a-030ab08a87d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:12.629548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2da647a8-5e79-4eec-b95a-030ab08a87d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:12.630815 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2da647a8-5e79-4eec-b95a-030ab08a87d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:12.631665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2da647a8-5e79-4eec-b95a-030ab08a87d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:12.632474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2da647a8-5e79-4eec-b95a-030ab08a87d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:12.692154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2da647a8-5e79-4eec-b95a-030ab08a87d9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.086375 Aug 07 01:21:12.693263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 591/1149] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:12 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:12.968831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5a8acf81-0acf-46cb-93c9-cc3a66a59bed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:12.969460 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5a8acf81-0acf-46cb-93c9-cc3a66a59bed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:12.977052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5a8acf81-0acf-46cb-93c9-cc3a66a59bed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:12.977753 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5a8acf81-0acf-46cb-93c9-cc3a66a59bed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:12.978201 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5a8acf81-0acf-46cb-93c9-cc3a66a59bed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:12.979051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5a8acf81-0acf-46cb-93c9-cc3a66a59bed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:12.979516 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5a8acf81-0acf-46cb-93c9-cc3a66a59bed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:12.980179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5a8acf81-0acf-46cb-93c9-cc3a66a59bed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:13.055794 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5a8acf81-0acf-46cb-93c9-cc3a66a59bed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:13.064845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5a8acf81-0acf-46cb-93c9-cc3a66a59bed tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.099895 Aug 07 01:21:13.065446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 559/1150] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:12 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:13.706114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e2940932-45dc-4494-8adc-529cc76c03ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:13.708922 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e2940932-45dc-4494-8adc-529cc76c03ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:13.715946 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e2940932-45dc-4494-8adc-529cc76c03ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:13.716469 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e2940932-45dc-4494-8adc-529cc76c03ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:13.716909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e2940932-45dc-4494-8adc-529cc76c03ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:13.717634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e2940932-45dc-4494-8adc-529cc76c03ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:13.718087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e2940932-45dc-4494-8adc-529cc76c03ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:13.718517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e2940932-45dc-4494-8adc-529cc76c03ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:13.753232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e2940932-45dc-4494-8adc-529cc76c03ab tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051168 Aug 07 01:21:13.754077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 592/1151] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:13 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:14.088002 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-41c24c9b-60cc-450f-8493-bc5dfc3e3b82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:14.088915 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-41c24c9b-60cc-450f-8493-bc5dfc3e3b82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:14.095838 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41c24c9b-60cc-450f-8493-bc5dfc3e3b82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:14.096458 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41c24c9b-60cc-450f-8493-bc5dfc3e3b82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:14.096792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41c24c9b-60cc-450f-8493-bc5dfc3e3b82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:14.097504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41c24c9b-60cc-450f-8493-bc5dfc3e3b82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:14.098564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41c24c9b-60cc-450f-8493-bc5dfc3e3b82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:14.098989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-41c24c9b-60cc-450f-8493-bc5dfc3e3b82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:14.186815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-41c24c9b-60cc-450f-8493-bc5dfc3e3b82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:14.196202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-41c24c9b-60cc-450f-8493-bc5dfc3e3b82 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.112342 Aug 07 01:21:14.197185 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 560/1152] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:14 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:14.770572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0c9026ed-71fc-4df3-b188-429218076c1c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:14.773570 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0c9026ed-71fc-4df3-b188-429218076c1c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:14.784560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c9026ed-71fc-4df3-b188-429218076c1c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:14.785321 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c9026ed-71fc-4df3-b188-429218076c1c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:14.786001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c9026ed-71fc-4df3-b188-429218076c1c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:14.787228 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c9026ed-71fc-4df3-b188-429218076c1c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:14.787991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c9026ed-71fc-4df3-b188-429218076c1c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:14.788703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c9026ed-71fc-4df3-b188-429218076c1c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:14.841992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0c9026ed-71fc-4df3-b188-429218076c1c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.078062 Aug 07 01:21:14.842905 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 593/1153] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:14 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:15.218214 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7c9a4917-05f4-4a51-8b11-39bbdd2e9770 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:15.219054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7c9a4917-05f4-4a51-8b11-39bbdd2e9770 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:15.225930 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7c9a4917-05f4-4a51-8b11-39bbdd2e9770 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:15.226422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7c9a4917-05f4-4a51-8b11-39bbdd2e9770 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:15.226856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7c9a4917-05f4-4a51-8b11-39bbdd2e9770 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:15.227578 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7c9a4917-05f4-4a51-8b11-39bbdd2e9770 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:15.228037 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7c9a4917-05f4-4a51-8b11-39bbdd2e9770 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:15.228496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7c9a4917-05f4-4a51-8b11-39bbdd2e9770 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:15.301939 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7c9a4917-05f4-4a51-8b11-39bbdd2e9770 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:15.312370 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7c9a4917-05f4-4a51-8b11-39bbdd2e9770 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.098053 Aug 07 01:21:15.313100 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 561/1154] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:15 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:15.860431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c489e98f-0be0-47ec-90b8-bdb77309a3a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:15.864416 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c489e98f-0be0-47ec-90b8-bdb77309a3a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:15.874775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c489e98f-0be0-47ec-90b8-bdb77309a3a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:15.875325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c489e98f-0be0-47ec-90b8-bdb77309a3a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:15.875776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c489e98f-0be0-47ec-90b8-bdb77309a3a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:15.876577 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c489e98f-0be0-47ec-90b8-bdb77309a3a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:15.877047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c489e98f-0be0-47ec-90b8-bdb77309a3a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:15.877554 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c489e98f-0be0-47ec-90b8-bdb77309a3a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:15.914396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c489e98f-0be0-47ec-90b8-bdb77309a3a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061705 Aug 07 01:21:15.915069 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 594/1155] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:15 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:16.339276 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ac972a5a-da33-4078-8551-d1fb6dd0f5d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:16.340270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ac972a5a-da33-4078-8551-d1fb6dd0f5d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:16.350933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ac972a5a-da33-4078-8551-d1fb6dd0f5d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:16.351534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ac972a5a-da33-4078-8551-d1fb6dd0f5d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:16.352025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ac972a5a-da33-4078-8551-d1fb6dd0f5d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:16.352828 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ac972a5a-da33-4078-8551-d1fb6dd0f5d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:16.353319 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ac972a5a-da33-4078-8551-d1fb6dd0f5d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:16.353789 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ac972a5a-da33-4078-8551-d1fb6dd0f5d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:16.445848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ac972a5a-da33-4078-8551-d1fb6dd0f5d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:16.458894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ac972a5a-da33-4078-8551-d1fb6dd0f5d5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.125532 Aug 07 01:21:16.459929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 562/1156] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:16 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:16.932997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e1ddc580-8ebd-4906-b99f-3951fff4b34c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:16.936723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e1ddc580-8ebd-4906-b99f-3951fff4b34c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:16.949433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e1ddc580-8ebd-4906-b99f-3951fff4b34c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:16.950845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e1ddc580-8ebd-4906-b99f-3951fff4b34c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:16.951370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e1ddc580-8ebd-4906-b99f-3951fff4b34c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:16.952691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e1ddc580-8ebd-4906-b99f-3951fff4b34c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:16.953524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e1ddc580-8ebd-4906-b99f-3951fff4b34c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:16.954305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e1ddc580-8ebd-4906-b99f-3951fff4b34c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:17.005026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e1ddc580-8ebd-4906-b99f-3951fff4b34c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080219 Aug 07 01:21:17.005921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 595/1157] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:16 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:17.483049 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5a762f5a-2252-49f3-965e-5eb3cb5a4a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:17.483796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5a762f5a-2252-49f3-965e-5eb3cb5a4a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:17.490933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5a762f5a-2252-49f3-965e-5eb3cb5a4a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:17.491462 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5a762f5a-2252-49f3-965e-5eb3cb5a4a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:17.491870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5a762f5a-2252-49f3-965e-5eb3cb5a4a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:17.492623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5a762f5a-2252-49f3-965e-5eb3cb5a4a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:17.493087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5a762f5a-2252-49f3-965e-5eb3cb5a4a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:17.493520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5a762f5a-2252-49f3-965e-5eb3cb5a4a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:17.562469 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5a762f5a-2252-49f3-965e-5eb3cb5a4a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:17.571333 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5a762f5a-2252-49f3-965e-5eb3cb5a4a3a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.092456 Aug 07 01:21:17.571943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 563/1158] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:17 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:18.028009 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-233fc5bd-8640-424e-bc55-cafb8c4407f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:18.031384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-233fc5bd-8640-424e-bc55-cafb8c4407f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:18.041821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-233fc5bd-8640-424e-bc55-cafb8c4407f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:18.042557 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-233fc5bd-8640-424e-bc55-cafb8c4407f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:18.043233 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-233fc5bd-8640-424e-bc55-cafb8c4407f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:18.044353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-233fc5bd-8640-424e-bc55-cafb8c4407f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:18.045086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-233fc5bd-8640-424e-bc55-cafb8c4407f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:18.045805 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-233fc5bd-8640-424e-bc55-cafb8c4407f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:18.111591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-233fc5bd-8640-424e-bc55-cafb8c4407f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.091252 Aug 07 01:21:18.112692 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 596/1159] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:18 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:18.594930 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f2a15653-bc71-42bc-816a-442ab4a8bfc3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:18.595701 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f2a15653-bc71-42bc-816a-442ab4a8bfc3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:18.604358 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2a15653-bc71-42bc-816a-442ab4a8bfc3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:18.604993 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2a15653-bc71-42bc-816a-442ab4a8bfc3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:18.605584 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2a15653-bc71-42bc-816a-442ab4a8bfc3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:18.606366 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2a15653-bc71-42bc-816a-442ab4a8bfc3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:18.606923 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2a15653-bc71-42bc-816a-442ab4a8bfc3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:18.607475 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2a15653-bc71-42bc-816a-442ab4a8bfc3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:18.705604 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f2a15653-bc71-42bc-816a-442ab4a8bfc3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:18.718751 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f2a15653-bc71-42bc-816a-442ab4a8bfc3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.127802 Aug 07 01:21:18.719979 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 564/1160] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:18 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:19.128882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cf0b1af5-61d1-4fa9-ab32-3c6b839b9dbe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:19.131541 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cf0b1af5-61d1-4fa9-ab32-3c6b839b9dbe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:19.138820 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cf0b1af5-61d1-4fa9-ab32-3c6b839b9dbe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:19.139323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cf0b1af5-61d1-4fa9-ab32-3c6b839b9dbe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:19.139783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cf0b1af5-61d1-4fa9-ab32-3c6b839b9dbe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:19.140661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cf0b1af5-61d1-4fa9-ab32-3c6b839b9dbe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:19.141134 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cf0b1af5-61d1-4fa9-ab32-3c6b839b9dbe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:19.141574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cf0b1af5-61d1-4fa9-ab32-3c6b839b9dbe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:19.146560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:21:19.181453 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cf0b1af5-61d1-4fa9-ab32-3c6b839b9dbe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056477 Aug 07 01:21:19.182458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 597/1161] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:19 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:19.754513 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c6af9a8b-2425-4a22-8434-6756d0ffe048 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:19.755209 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c6af9a8b-2425-4a22-8434-6756d0ffe048 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:19.762614 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6af9a8b-2425-4a22-8434-6756d0ffe048 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:19.763146 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6af9a8b-2425-4a22-8434-6756d0ffe048 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:19.763625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6af9a8b-2425-4a22-8434-6756d0ffe048 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:19.764473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6af9a8b-2425-4a22-8434-6756d0ffe048 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:19.765035 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6af9a8b-2425-4a22-8434-6756d0ffe048 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:19.765501 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6af9a8b-2425-4a22-8434-6756d0ffe048 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:19.839742 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c6af9a8b-2425-4a22-8434-6756d0ffe048 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:19.849548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c6af9a8b-2425-4a22-8434-6756d0ffe048 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.099081 Aug 07 01:21:19.850516 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 565/1162] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:19 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:20.196705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c8febba7-8ce2-466f-96df-d20e850e1868 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:20.199225 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c8febba7-8ce2-466f-96df-d20e850e1868 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:20.208438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8febba7-8ce2-466f-96df-d20e850e1868 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:20.209196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8febba7-8ce2-466f-96df-d20e850e1868 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:20.209782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8febba7-8ce2-466f-96df-d20e850e1868 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:20.210748 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8febba7-8ce2-466f-96df-d20e850e1868 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:20.211264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8febba7-8ce2-466f-96df-d20e850e1868 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:20.211714 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8febba7-8ce2-466f-96df-d20e850e1868 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:20.249005 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c8febba7-8ce2-466f-96df-d20e850e1868 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057748 Aug 07 01:21:20.249821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 598/1163] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:20 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:20.871791 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4a266072-2998-44f3-b94b-e6effe8eae8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:20.872616 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4a266072-2998-44f3-b94b-e6effe8eae8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:20.879681 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a266072-2998-44f3-b94b-e6effe8eae8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:20.880130 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a266072-2998-44f3-b94b-e6effe8eae8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:20.880898 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a266072-2998-44f3-b94b-e6effe8eae8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:20.881410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a266072-2998-44f3-b94b-e6effe8eae8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:20.881868 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a266072-2998-44f3-b94b-e6effe8eae8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:20.882301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a266072-2998-44f3-b94b-e6effe8eae8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:20.951571 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4a266072-2998-44f3-b94b-e6effe8eae8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:20.960338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4a266072-2998-44f3-b94b-e6effe8eae8d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.092339 Aug 07 01:21:20.960928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 566/1164] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:20 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:21.266824 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1e3dd518-3bd7-4949-8157-8cb203a95af1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:21.269161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1e3dd518-3bd7-4949-8157-8cb203a95af1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:21.277002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e3dd518-3bd7-4949-8157-8cb203a95af1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:21.277800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e3dd518-3bd7-4949-8157-8cb203a95af1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:21.277995 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e3dd518-3bd7-4949-8157-8cb203a95af1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:21.278674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e3dd518-3bd7-4949-8157-8cb203a95af1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:21.279153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e3dd518-3bd7-4949-8157-8cb203a95af1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:21.279585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e3dd518-3bd7-4949-8157-8cb203a95af1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:21.315537 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1e3dd518-3bd7-4949-8157-8cb203a95af1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052889 Aug 07 01:21:21.316784 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 599/1165] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:21 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:21.984482 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f2e61db3-510f-4cc4-bbc6-a929484de29d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:21.985192 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f2e61db3-510f-4cc4-bbc6-a929484de29d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:21.992942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2e61db3-510f-4cc4-bbc6-a929484de29d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:21.993488 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2e61db3-510f-4cc4-bbc6-a929484de29d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:21.993997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2e61db3-510f-4cc4-bbc6-a929484de29d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:21.994845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2e61db3-510f-4cc4-bbc6-a929484de29d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:21.995356 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2e61db3-510f-4cc4-bbc6-a929484de29d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:21.995833 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2e61db3-510f-4cc4-bbc6-a929484de29d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:22.068610 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f2e61db3-510f-4cc4-bbc6-a929484de29d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:22.078063 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f2e61db3-510f-4cc4-bbc6-a929484de29d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.099716 Aug 07 01:21:22.078660 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 567/1166] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:21 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:22.329556 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a8aa45ab-3c91-47d0-8018-0df9b521fa88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:22.332177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a8aa45ab-3c91-47d0-8018-0df9b521fa88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:22.340090 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8aa45ab-3c91-47d0-8018-0df9b521fa88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:22.340749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8aa45ab-3c91-47d0-8018-0df9b521fa88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:22.341279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8aa45ab-3c91-47d0-8018-0df9b521fa88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:22.342219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8aa45ab-3c91-47d0-8018-0df9b521fa88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:22.342698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8aa45ab-3c91-47d0-8018-0df9b521fa88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:22.343140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8aa45ab-3c91-47d0-8018-0df9b521fa88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:22.377620 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a8aa45ab-3c91-47d0-8018-0df9b521fa88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052784 Aug 07 01:21:22.378218 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 600/1167] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:22 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:23.113191 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8521e04d-703e-473a-b2be-7f865edca52f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:23.114035 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8521e04d-703e-473a-b2be-7f865edca52f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:23.122466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8521e04d-703e-473a-b2be-7f865edca52f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:23.122967 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8521e04d-703e-473a-b2be-7f865edca52f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:23.123408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8521e04d-703e-473a-b2be-7f865edca52f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:23.124150 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8521e04d-703e-473a-b2be-7f865edca52f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:23.124679 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8521e04d-703e-473a-b2be-7f865edca52f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:23.125115 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8521e04d-703e-473a-b2be-7f865edca52f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:23.218891 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8521e04d-703e-473a-b2be-7f865edca52f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:23.227752 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8521e04d-703e-473a-b2be-7f865edca52f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.118657 Aug 07 01:21:23.228367 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 568/1168] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:23 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:23.390955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-15e32ced-e96b-44f9-a121-488e8e4a3475 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:23.393310 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-15e32ced-e96b-44f9-a121-488e8e4a3475 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:23.401483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15e32ced-e96b-44f9-a121-488e8e4a3475 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:23.401988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15e32ced-e96b-44f9-a121-488e8e4a3475 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:23.402534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15e32ced-e96b-44f9-a121-488e8e4a3475 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:23.403298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15e32ced-e96b-44f9-a121-488e8e4a3475 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:23.403753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15e32ced-e96b-44f9-a121-488e8e4a3475 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:23.404183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15e32ced-e96b-44f9-a121-488e8e4a3475 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:23.439066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-15e32ced-e96b-44f9-a121-488e8e4a3475 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052519 Aug 07 01:21:23.439662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 601/1169] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:23 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:24.262722 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f77b7f8f-f48b-4205-88c2-9bf6ec89ba0e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:24.263384 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f77b7f8f-f48b-4205-88c2-9bf6ec89ba0e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:24.271792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f77b7f8f-f48b-4205-88c2-9bf6ec89ba0e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:24.272448 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f77b7f8f-f48b-4205-88c2-9bf6ec89ba0e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:24.273002 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f77b7f8f-f48b-4205-88c2-9bf6ec89ba0e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:24.273865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f77b7f8f-f48b-4205-88c2-9bf6ec89ba0e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:24.274633 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f77b7f8f-f48b-4205-88c2-9bf6ec89ba0e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:24.275300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f77b7f8f-f48b-4205-88c2-9bf6ec89ba0e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:24.350469 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f77b7f8f-f48b-4205-88c2-9bf6ec89ba0e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:24.363777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f77b7f8f-f48b-4205-88c2-9bf6ec89ba0e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.104498 Aug 07 01:21:24.364884 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 569/1170] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:24 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:24.453203 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6102006b-2559-43af-aec1-e025ddc7a2e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:24.455515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6102006b-2559-43af-aec1-e025ddc7a2e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:24.464334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6102006b-2559-43af-aec1-e025ddc7a2e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:24.465185 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6102006b-2559-43af-aec1-e025ddc7a2e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:24.465649 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6102006b-2559-43af-aec1-e025ddc7a2e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:24.466400 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6102006b-2559-43af-aec1-e025ddc7a2e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:24.467101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6102006b-2559-43af-aec1-e025ddc7a2e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:24.467730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6102006b-2559-43af-aec1-e025ddc7a2e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:24.501673 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6102006b-2559-43af-aec1-e025ddc7a2e9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053037 Aug 07 01:21:24.502261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 602/1171] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:24 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:25.401159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7cc5f16f-6a33-4c84-82b7-19329ee5711e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:25.402347 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7cc5f16f-6a33-4c84-82b7-19329ee5711e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:25.413388 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7cc5f16f-6a33-4c84-82b7-19329ee5711e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:25.414252 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7cc5f16f-6a33-4c84-82b7-19329ee5711e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:25.415071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7cc5f16f-6a33-4c84-82b7-19329ee5711e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:25.416796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7cc5f16f-6a33-4c84-82b7-19329ee5711e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:25.417644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7cc5f16f-6a33-4c84-82b7-19329ee5711e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:25.418413 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7cc5f16f-6a33-4c84-82b7-19329ee5711e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:25.515195 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7cc5f16f-6a33-4c84-82b7-19329ee5711e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:25.517672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b5f3f811-65ab-4760-bf8e-392312322915 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:25.520885 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b5f3f811-65ab-4760-bf8e-392312322915 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:25.524216 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7cc5f16f-6a33-4c84-82b7-19329ee5711e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.130137 Aug 07 01:21:25.524923 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 570/1172] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:25 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:25.530001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5f3f811-65ab-4760-bf8e-392312322915 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:25.530655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5f3f811-65ab-4760-bf8e-392312322915 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:25.531213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5f3f811-65ab-4760-bf8e-392312322915 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:25.532151 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5f3f811-65ab-4760-bf8e-392312322915 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:25.532713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5f3f811-65ab-4760-bf8e-392312322915 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:25.533372 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5f3f811-65ab-4760-bf8e-392312322915 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:25.569175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b5f3f811-65ab-4760-bf8e-392312322915 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057709 Aug 07 01:21:25.569790 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 603/1173] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:25 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:26.548095 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ffc738bc-de80-4fd9-b154-6253d106b1c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:26.548756 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ffc738bc-de80-4fd9-b154-6253d106b1c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:26.556695 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffc738bc-de80-4fd9-b154-6253d106b1c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:26.557224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffc738bc-de80-4fd9-b154-6253d106b1c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:26.557641 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffc738bc-de80-4fd9-b154-6253d106b1c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:26.558377 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffc738bc-de80-4fd9-b154-6253d106b1c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:26.558839 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffc738bc-de80-4fd9-b154-6253d106b1c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:26.559275 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffc738bc-de80-4fd9-b154-6253d106b1c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:26.587498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e5e7c6e1-a05c-468b-acac-7ea9bfc8fd62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:26.591300 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e5e7c6e1-a05c-468b-acac-7ea9bfc8fd62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:26.604296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5e7c6e1-a05c-468b-acac-7ea9bfc8fd62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:26.605164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5e7c6e1-a05c-468b-acac-7ea9bfc8fd62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:26.605929 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5e7c6e1-a05c-468b-acac-7ea9bfc8fd62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:26.607201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5e7c6e1-a05c-468b-acac-7ea9bfc8fd62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:26.608048 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5e7c6e1-a05c-468b-acac-7ea9bfc8fd62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:26.608853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5e7c6e1-a05c-468b-acac-7ea9bfc8fd62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:26.635668 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ffc738bc-de80-4fd9-b154-6253d106b1c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:26.645400 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ffc738bc-de80-4fd9-b154-6253d106b1c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.101321 Aug 07 01:21:26.646247 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 571/1174] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:26 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:26.668211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e5e7c6e1-a05c-468b-acac-7ea9bfc8fd62 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.088709 Aug 07 01:21:26.669318 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 604/1175] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:26 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:27.671159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f432db4f-9fe9-4bb2-b443-58cf56486266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:27.672193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f432db4f-9fe9-4bb2-b443-58cf56486266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:27.683623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f432db4f-9fe9-4bb2-b443-58cf56486266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:27.684492 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f432db4f-9fe9-4bb2-b443-58cf56486266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:27.684658 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0b969dc6-0df3-4ddc-a77f-8a5bd53b12a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:27.685265 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f432db4f-9fe9-4bb2-b443-58cf56486266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:27.686540 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f432db4f-9fe9-4bb2-b443-58cf56486266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:27.686762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0b969dc6-0df3-4ddc-a77f-8a5bd53b12a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:27.687342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f432db4f-9fe9-4bb2-b443-58cf56486266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:27.688105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f432db4f-9fe9-4bb2-b443-58cf56486266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:27.694422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b969dc6-0df3-4ddc-a77f-8a5bd53b12a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:27.694920 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b969dc6-0df3-4ddc-a77f-8a5bd53b12a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:27.695759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b969dc6-0df3-4ddc-a77f-8a5bd53b12a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:27.696519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b969dc6-0df3-4ddc-a77f-8a5bd53b12a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:27.697012 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b969dc6-0df3-4ddc-a77f-8a5bd53b12a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:27.697543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b969dc6-0df3-4ddc-a77f-8a5bd53b12a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:27.730027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:21:27.731425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0b969dc6-0df3-4ddc-a77f-8a5bd53b12a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050852 Aug 07 01:21:27.739385 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 605/1176] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:27 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:27.791952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f432db4f-9fe9-4bb2-b443-58cf56486266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:27.801678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f432db4f-9fe9-4bb2-b443-58cf56486266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.137534 Aug 07 01:21:27.802290 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 572/1177] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:27 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:28.748634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-83939d6e-49dc-483d-84ff-58fb78a6c764 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:28.750822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-83939d6e-49dc-483d-84ff-58fb78a6c764 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:28.757863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83939d6e-49dc-483d-84ff-58fb78a6c764 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:28.758375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83939d6e-49dc-483d-84ff-58fb78a6c764 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:28.758813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83939d6e-49dc-483d-84ff-58fb78a6c764 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:28.759524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83939d6e-49dc-483d-84ff-58fb78a6c764 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:28.760125 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83939d6e-49dc-483d-84ff-58fb78a6c764 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:28.760584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-83939d6e-49dc-483d-84ff-58fb78a6c764 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:28.796071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-83939d6e-49dc-483d-84ff-58fb78a6c764 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052428 Aug 07 01:21:28.796899 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 606/1178] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:28 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:28.828714 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d72459ce-93f6-4e42-bf29-96b21bc0761b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:28.829516 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d72459ce-93f6-4e42-bf29-96b21bc0761b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:28.838656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d72459ce-93f6-4e42-bf29-96b21bc0761b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:28.839170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d72459ce-93f6-4e42-bf29-96b21bc0761b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:28.839690 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d72459ce-93f6-4e42-bf29-96b21bc0761b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:28.840899 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d72459ce-93f6-4e42-bf29-96b21bc0761b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:28.841376 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d72459ce-93f6-4e42-bf29-96b21bc0761b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:28.842110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d72459ce-93f6-4e42-bf29-96b21bc0761b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:28.910684 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d72459ce-93f6-4e42-bf29-96b21bc0761b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:28.919948 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d72459ce-93f6-4e42-bf29-96b21bc0761b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.096405 Aug 07 01:21:28.921172 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 573/1179] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:28 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:29.809781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f18f4f7d-f943-49f3-8e9f-a63638dfa7f1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:29.812077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f18f4f7d-f943-49f3-8e9f-a63638dfa7f1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:29.820045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f18f4f7d-f943-49f3-8e9f-a63638dfa7f1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:29.820763 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f18f4f7d-f943-49f3-8e9f-a63638dfa7f1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:29.821442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f18f4f7d-f943-49f3-8e9f-a63638dfa7f1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:29.822604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f18f4f7d-f943-49f3-8e9f-a63638dfa7f1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:29.823150 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f18f4f7d-f943-49f3-8e9f-a63638dfa7f1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:29.823634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f18f4f7d-f943-49f3-8e9f-a63638dfa7f1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:29.860533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f18f4f7d-f943-49f3-8e9f-a63638dfa7f1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054851 Aug 07 01:21:29.861110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 607/1180] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:29 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:29.942761 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-59e25ace-b058-470d-89c1-2555aa820ca0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:29.943475 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-59e25ace-b058-470d-89c1-2555aa820ca0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:29.950567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59e25ace-b058-470d-89c1-2555aa820ca0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:29.951064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59e25ace-b058-470d-89c1-2555aa820ca0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:29.951507 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59e25ace-b058-470d-89c1-2555aa820ca0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:29.952266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59e25ace-b058-470d-89c1-2555aa820ca0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:29.952739 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59e25ace-b058-470d-89c1-2555aa820ca0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:29.953170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-59e25ace-b058-470d-89c1-2555aa820ca0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:30.041387 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-59e25ace-b058-470d-89c1-2555aa820ca0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:30.050737 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-59e25ace-b058-470d-89c1-2555aa820ca0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.111946 Aug 07 01:21:30.051713 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 574/1181] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:29 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:30.874545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b525ec8f-adf2-4694-ac23-366ef51836a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:30.877016 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b525ec8f-adf2-4694-ac23-366ef51836a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:30.885223 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b525ec8f-adf2-4694-ac23-366ef51836a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:30.885724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b525ec8f-adf2-4694-ac23-366ef51836a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:30.886162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b525ec8f-adf2-4694-ac23-366ef51836a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:30.886983 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b525ec8f-adf2-4694-ac23-366ef51836a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:30.887442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b525ec8f-adf2-4694-ac23-366ef51836a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:30.887874 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b525ec8f-adf2-4694-ac23-366ef51836a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:30.922863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b525ec8f-adf2-4694-ac23-366ef51836a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052551 Aug 07 01:21:30.923577 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 608/1182] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:30 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:31.073853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b6243a34-7cb1-449b-8aff-a3aedcc252ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:31.074944 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b6243a34-7cb1-449b-8aff-a3aedcc252ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:31.082240 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b6243a34-7cb1-449b-8aff-a3aedcc252ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:31.082733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b6243a34-7cb1-449b-8aff-a3aedcc252ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:31.083165 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b6243a34-7cb1-449b-8aff-a3aedcc252ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:31.083886 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b6243a34-7cb1-449b-8aff-a3aedcc252ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:31.084367 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b6243a34-7cb1-449b-8aff-a3aedcc252ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:31.084792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b6243a34-7cb1-449b-8aff-a3aedcc252ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:31.157664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b6243a34-7cb1-449b-8aff-a3aedcc252ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:31.166862 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b6243a34-7cb1-449b-8aff-a3aedcc252ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097721 Aug 07 01:21:31.167454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 575/1183] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:31 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:31.942478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-efcdf97e-c5c9-4c85-8f29-851a100a92b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:31.946316 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-efcdf97e-c5c9-4c85-8f29-851a100a92b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:31.958434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-efcdf97e-c5c9-4c85-8f29-851a100a92b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:31.959305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-efcdf97e-c5c9-4c85-8f29-851a100a92b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:31.960106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-efcdf97e-c5c9-4c85-8f29-851a100a92b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:31.961422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-efcdf97e-c5c9-4c85-8f29-851a100a92b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:31.962216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-efcdf97e-c5c9-4c85-8f29-851a100a92b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:31.962988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-efcdf97e-c5c9-4c85-8f29-851a100a92b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:31.999831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-efcdf97e-c5c9-4c85-8f29-851a100a92b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065818 Aug 07 01:21:32.000628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 609/1184] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:31 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:32.199695 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-431b74f6-a7e6-4b0b-9b55-63bf9e401406 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:32.200636 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-431b74f6-a7e6-4b0b-9b55-63bf9e401406 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:32.208403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-431b74f6-a7e6-4b0b-9b55-63bf9e401406 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:32.208774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-431b74f6-a7e6-4b0b-9b55-63bf9e401406 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:32.209213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-431b74f6-a7e6-4b0b-9b55-63bf9e401406 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:32.209929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-431b74f6-a7e6-4b0b-9b55-63bf9e401406 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:32.210387 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-431b74f6-a7e6-4b0b-9b55-63bf9e401406 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:32.210805 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-431b74f6-a7e6-4b0b-9b55-63bf9e401406 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:32.287265 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-431b74f6-a7e6-4b0b-9b55-63bf9e401406 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:32.302336 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-431b74f6-a7e6-4b0b-9b55-63bf9e401406 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.106593 Aug 07 01:21:32.304192 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 576/1185] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:32 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:33.013538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-db2a4bad-00e5-4e09-b594-46012305b70d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:33.015781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-db2a4bad-00e5-4e09-b594-46012305b70d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:33.018249 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=56 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:21:33.024710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-db2a4bad-00e5-4e09-b594-46012305b70d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:33.025213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-db2a4bad-00e5-4e09-b594-46012305b70d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:33.025660 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-db2a4bad-00e5-4e09-b594-46012305b70d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:33.026392 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-db2a4bad-00e5-4e09-b594-46012305b70d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:33.026854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-db2a4bad-00e5-4e09-b594-46012305b70d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:33.027287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-db2a4bad-00e5-4e09-b594-46012305b70d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:33.062308 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-db2a4bad-00e5-4e09-b594-46012305b70d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052916 Aug 07 01:21:33.062975 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 610/1186] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:33 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:33.337285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d2b498d4-c3a7-4484-acb7-d3872335dde9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:33.338136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d2b498d4-c3a7-4484-acb7-d3872335dde9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:33.345417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d2b498d4-c3a7-4484-acb7-d3872335dde9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:33.345914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d2b498d4-c3a7-4484-acb7-d3872335dde9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:33.346391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d2b498d4-c3a7-4484-acb7-d3872335dde9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:33.348058 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d2b498d4-c3a7-4484-acb7-d3872335dde9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:33.348058 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d2b498d4-c3a7-4484-acb7-d3872335dde9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:33.348485 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d2b498d4-c3a7-4484-acb7-d3872335dde9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:33.452044 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d2b498d4-c3a7-4484-acb7-d3872335dde9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:33.466633 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d2b498d4-c3a7-4484-acb7-d3872335dde9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.132930 Aug 07 01:21:33.467679 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 577/1187] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:33 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:34.076230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-248db526-68e6-40b4-a1bc-8ea2af8ee784 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:34.078503 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-248db526-68e6-40b4-a1bc-8ea2af8ee784 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:34.087350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-248db526-68e6-40b4-a1bc-8ea2af8ee784 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:34.088163 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-248db526-68e6-40b4-a1bc-8ea2af8ee784 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:34.089058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-248db526-68e6-40b4-a1bc-8ea2af8ee784 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:34.090061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-248db526-68e6-40b4-a1bc-8ea2af8ee784 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:34.090535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-248db526-68e6-40b4-a1bc-8ea2af8ee784 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:34.090973 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-248db526-68e6-40b4-a1bc-8ea2af8ee784 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:34.125180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-248db526-68e6-40b4-a1bc-8ea2af8ee784 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053288 Aug 07 01:21:34.125673 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 611/1188] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:34 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:34.491174 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0d211943-5024-43e3-8369-71341d5c28a5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:34.492051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0d211943-5024-43e3-8369-71341d5c28a5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:34.494372 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=54 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:21:34.500766 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d211943-5024-43e3-8369-71341d5c28a5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:34.501311 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d211943-5024-43e3-8369-71341d5c28a5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:34.501757 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d211943-5024-43e3-8369-71341d5c28a5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:34.502490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d211943-5024-43e3-8369-71341d5c28a5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:34.502949 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d211943-5024-43e3-8369-71341d5c28a5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:34.503495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d211943-5024-43e3-8369-71341d5c28a5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:34.584043 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0d211943-5024-43e3-8369-71341d5c28a5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:34.593210 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0d211943-5024-43e3-8369-71341d5c28a5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.106923 Aug 07 01:21:34.593943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 578/1189] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:34 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:35.138788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e3cc39cb-c42c-4ca1-bf86-86160e2e332e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:35.141122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e3cc39cb-c42c-4ca1-bf86-86160e2e332e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:35.148910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3cc39cb-c42c-4ca1-bf86-86160e2e332e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:35.149411 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3cc39cb-c42c-4ca1-bf86-86160e2e332e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:35.149863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3cc39cb-c42c-4ca1-bf86-86160e2e332e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:35.150598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3cc39cb-c42c-4ca1-bf86-86160e2e332e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:35.151063 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3cc39cb-c42c-4ca1-bf86-86160e2e332e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:35.151537 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3cc39cb-c42c-4ca1-bf86-86160e2e332e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:35.187582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e3cc39cb-c42c-4ca1-bf86-86160e2e332e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053054 Aug 07 01:21:35.188399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 612/1190] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:35 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:35.615808 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-37aeb0aa-921a-4b12-b860-d2d4c8daf8fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:35.616469 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-37aeb0aa-921a-4b12-b860-d2d4c8daf8fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:35.623911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37aeb0aa-921a-4b12-b860-d2d4c8daf8fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:35.624444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37aeb0aa-921a-4b12-b860-d2d4c8daf8fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:35.624879 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37aeb0aa-921a-4b12-b860-d2d4c8daf8fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:35.625612 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37aeb0aa-921a-4b12-b860-d2d4c8daf8fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:35.626167 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37aeb0aa-921a-4b12-b860-d2d4c8daf8fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:35.626595 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37aeb0aa-921a-4b12-b860-d2d4c8daf8fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:35.704224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-37aeb0aa-921a-4b12-b860-d2d4c8daf8fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:35.713172 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-37aeb0aa-921a-4b12-b860-d2d4c8daf8fb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.101549 Aug 07 01:21:35.713817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 579/1191] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:35 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:36.202542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a1e3fa7f-b096-41e3-a367-1817291ccd2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:36.204958 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a1e3fa7f-b096-41e3-a367-1817291ccd2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:36.213375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1e3fa7f-b096-41e3-a367-1817291ccd2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:36.213874 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1e3fa7f-b096-41e3-a367-1817291ccd2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:36.214751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1e3fa7f-b096-41e3-a367-1817291ccd2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:36.215567 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1e3fa7f-b096-41e3-a367-1817291ccd2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:36.216035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1e3fa7f-b096-41e3-a367-1817291ccd2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:36.216510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1e3fa7f-b096-41e3-a367-1817291ccd2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:36.245938 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:21:36.252273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a1e3fa7f-b096-41e3-a367-1817291ccd2c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054365 Aug 07 01:21:36.252782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 613/1192] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:36 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:36.736546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-be50275f-fb76-4b77-a94e-4116fd573876 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:36.737219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-be50275f-fb76-4b77-a94e-4116fd573876 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:36.744866 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be50275f-fb76-4b77-a94e-4116fd573876 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:36.745332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be50275f-fb76-4b77-a94e-4116fd573876 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:36.745785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be50275f-fb76-4b77-a94e-4116fd573876 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:36.746561 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be50275f-fb76-4b77-a94e-4116fd573876 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:36.747022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be50275f-fb76-4b77-a94e-4116fd573876 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:36.747461 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be50275f-fb76-4b77-a94e-4116fd573876 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:36.824185 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-be50275f-fb76-4b77-a94e-4116fd573876 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:36.838969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-be50275f-fb76-4b77-a94e-4116fd573876 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.106259 Aug 07 01:21:36.840017 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 580/1193] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:36 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:37.265581 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-250416ad-03c5-4f46-8778-1e800c3baa82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:37.267869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-250416ad-03c5-4f46-8778-1e800c3baa82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:37.276565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-250416ad-03c5-4f46-8778-1e800c3baa82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:37.277250 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-250416ad-03c5-4f46-8778-1e800c3baa82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:37.277807 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-250416ad-03c5-4f46-8778-1e800c3baa82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:37.278895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-250416ad-03c5-4f46-8778-1e800c3baa82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:37.279680 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-250416ad-03c5-4f46-8778-1e800c3baa82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:37.280270 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-250416ad-03c5-4f46-8778-1e800c3baa82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:37.317216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-250416ad-03c5-4f46-8778-1e800c3baa82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055697 Aug 07 01:21:37.317822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 614/1194] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:37 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:37.870285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-be307be9-b0b9-4c4f-904c-53f7309a9fe7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:37.871111 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-be307be9-b0b9-4c4f-904c-53f7309a9fe7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:37.878947 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be307be9-b0b9-4c4f-904c-53f7309a9fe7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:37.879435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be307be9-b0b9-4c4f-904c-53f7309a9fe7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:37.879875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be307be9-b0b9-4c4f-904c-53f7309a9fe7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:37.880677 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be307be9-b0b9-4c4f-904c-53f7309a9fe7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:37.881130 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be307be9-b0b9-4c4f-904c-53f7309a9fe7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:37.881562 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-be307be9-b0b9-4c4f-904c-53f7309a9fe7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:37.961486 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-be307be9-b0b9-4c4f-904c-53f7309a9fe7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:37.969783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-be307be9-b0b9-4c4f-904c-53f7309a9fe7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.103747 Aug 07 01:21:37.970378 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 581/1195] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:37 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:38.330451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e9e7ed1f-20c1-48da-b267-2ac26914a802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:38.333461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e9e7ed1f-20c1-48da-b267-2ac26914a802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:38.342427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9e7ed1f-20c1-48da-b267-2ac26914a802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:38.343045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9e7ed1f-20c1-48da-b267-2ac26914a802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:38.343687 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9e7ed1f-20c1-48da-b267-2ac26914a802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:38.344495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9e7ed1f-20c1-48da-b267-2ac26914a802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:38.344961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9e7ed1f-20c1-48da-b267-2ac26914a802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:38.345397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9e7ed1f-20c1-48da-b267-2ac26914a802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:38.380311 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e9e7ed1f-20c1-48da-b267-2ac26914a802 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053974 Aug 07 01:21:38.380914 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 615/1196] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:38 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:39.005865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-30620bdb-eb00-4388-9845-108d0aac79dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:39.006553 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-30620bdb-eb00-4388-9845-108d0aac79dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:39.014225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-30620bdb-eb00-4388-9845-108d0aac79dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:39.014728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-30620bdb-eb00-4388-9845-108d0aac79dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:39.015170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-30620bdb-eb00-4388-9845-108d0aac79dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:39.015945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-30620bdb-eb00-4388-9845-108d0aac79dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:39.016440 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-30620bdb-eb00-4388-9845-108d0aac79dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:39.016877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-30620bdb-eb00-4388-9845-108d0aac79dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:39.088595 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-30620bdb-eb00-4388-9845-108d0aac79dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:39.097647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-30620bdb-eb00-4388-9845-108d0aac79dc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.096006 Aug 07 01:21:39.098506 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 582/1197] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:39 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:39.393417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9520f71d-2563-4df4-96c9-1d7c074786f0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:39.395742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9520f71d-2563-4df4-96c9-1d7c074786f0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:39.402645 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9520f71d-2563-4df4-96c9-1d7c074786f0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:39.403225 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9520f71d-2563-4df4-96c9-1d7c074786f0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:39.403723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9520f71d-2563-4df4-96c9-1d7c074786f0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:39.404554 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9520f71d-2563-4df4-96c9-1d7c074786f0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:39.405157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9520f71d-2563-4df4-96c9-1d7c074786f0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:39.405676 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9520f71d-2563-4df4-96c9-1d7c074786f0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:39.443301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9520f71d-2563-4df4-96c9-1d7c074786f0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052682 Aug 07 01:21:39.443965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 616/1198] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:39 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:40.120639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-703a4849-6e55-4da2-a154-973faf942af6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:40.121343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-703a4849-6e55-4da2-a154-973faf942af6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:40.130054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-703a4849-6e55-4da2-a154-973faf942af6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:40.130922 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-703a4849-6e55-4da2-a154-973faf942af6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:40.131678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-703a4849-6e55-4da2-a154-973faf942af6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:40.132918 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-703a4849-6e55-4da2-a154-973faf942af6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:40.133717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-703a4849-6e55-4da2-a154-973faf942af6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:40.134465 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-703a4849-6e55-4da2-a154-973faf942af6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:40.208816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-703a4849-6e55-4da2-a154-973faf942af6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:40.216923 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-703a4849-6e55-4da2-a154-973faf942af6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100201 Aug 07 01:21:40.217576 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 583/1199] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:40 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:40.456713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0149397a-422a-411d-8f92-d0267fd31527 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:40.458959 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0149397a-422a-411d-8f92-d0267fd31527 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:40.466219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0149397a-422a-411d-8f92-d0267fd31527 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:40.466753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0149397a-422a-411d-8f92-d0267fd31527 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:40.467200 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0149397a-422a-411d-8f92-d0267fd31527 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:40.467937 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0149397a-422a-411d-8f92-d0267fd31527 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:40.468429 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0149397a-422a-411d-8f92-d0267fd31527 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:40.468861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0149397a-422a-411d-8f92-d0267fd31527 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:40.507588 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0149397a-422a-411d-8f92-d0267fd31527 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054890 Aug 07 01:21:40.508531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 617/1200] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:40 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:41.238712 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-02baa5cc-c056-4651-a8bf-eaef65a1fe23 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:41.239431 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-02baa5cc-c056-4651-a8bf-eaef65a1fe23 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:41.246907 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02baa5cc-c056-4651-a8bf-eaef65a1fe23 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:41.247454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02baa5cc-c056-4651-a8bf-eaef65a1fe23 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:41.248059 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02baa5cc-c056-4651-a8bf-eaef65a1fe23 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:41.248912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02baa5cc-c056-4651-a8bf-eaef65a1fe23 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:41.249416 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02baa5cc-c056-4651-a8bf-eaef65a1fe23 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:41.249890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02baa5cc-c056-4651-a8bf-eaef65a1fe23 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:41.330731 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-02baa5cc-c056-4651-a8bf-eaef65a1fe23 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:41.345588 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-02baa5cc-c056-4651-a8bf-eaef65a1fe23 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.110950 Aug 07 01:21:41.346660 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 584/1201] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:41 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:41.523786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cd3ae255-6a29-4f76-9f22-83d49a1dbe9a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:41.526306 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cd3ae255-6a29-4f76-9f22-83d49a1dbe9a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:41.533732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd3ae255-6a29-4f76-9f22-83d49a1dbe9a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:41.534672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd3ae255-6a29-4f76-9f22-83d49a1dbe9a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:41.535166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd3ae255-6a29-4f76-9f22-83d49a1dbe9a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:41.535997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd3ae255-6a29-4f76-9f22-83d49a1dbe9a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:41.536719 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd3ae255-6a29-4f76-9f22-83d49a1dbe9a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:41.537231 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd3ae255-6a29-4f76-9f22-83d49a1dbe9a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:41.575120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cd3ae255-6a29-4f76-9f22-83d49a1dbe9a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055909 Aug 07 01:21:41.575758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 618/1202] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:41 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:42.380281 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c0b8c2ed-35df-4a49-b5bc-764ceb47c8b1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:42.380966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c0b8c2ed-35df-4a49-b5bc-764ceb47c8b1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:42.388307 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c0b8c2ed-35df-4a49-b5bc-764ceb47c8b1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:42.388814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c0b8c2ed-35df-4a49-b5bc-764ceb47c8b1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:42.389247 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c0b8c2ed-35df-4a49-b5bc-764ceb47c8b1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:42.390043 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c0b8c2ed-35df-4a49-b5bc-764ceb47c8b1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:42.390509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c0b8c2ed-35df-4a49-b5bc-764ceb47c8b1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:42.390945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c0b8c2ed-35df-4a49-b5bc-764ceb47c8b1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:42.467812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c0b8c2ed-35df-4a49-b5bc-764ceb47c8b1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:42.477282 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c0b8c2ed-35df-4a49-b5bc-764ceb47c8b1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100999 Aug 07 01:21:42.478139 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 585/1203] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:42 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:42.589281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d85ade2a-ef8f-4eac-a727-f9ff4475c577 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:42.592427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d85ade2a-ef8f-4eac-a727-f9ff4475c577 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:42.599943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d85ade2a-ef8f-4eac-a727-f9ff4475c577 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:42.600488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d85ade2a-ef8f-4eac-a727-f9ff4475c577 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:42.600941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d85ade2a-ef8f-4eac-a727-f9ff4475c577 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:42.601722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d85ade2a-ef8f-4eac-a727-f9ff4475c577 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:42.602194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d85ade2a-ef8f-4eac-a727-f9ff4475c577 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:42.602631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d85ade2a-ef8f-4eac-a727-f9ff4475c577 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:42.643057 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d85ade2a-ef8f-4eac-a727-f9ff4475c577 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058402 Aug 07 01:21:42.644307 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 619/1204] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:42 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:43.499440 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-79f7f53f-5d48-432d-863d-72720746da25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:43.500370 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-79f7f53f-5d48-432d-863d-72720746da25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:43.507528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-79f7f53f-5d48-432d-863d-72720746da25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:43.508023 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-79f7f53f-5d48-432d-863d-72720746da25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:43.508495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-79f7f53f-5d48-432d-863d-72720746da25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:43.509229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-79f7f53f-5d48-432d-863d-72720746da25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:43.509870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-79f7f53f-5d48-432d-863d-72720746da25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:43.510406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-79f7f53f-5d48-432d-863d-72720746da25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:43.594065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-79f7f53f-5d48-432d-863d-72720746da25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:43.606084 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-79f7f53f-5d48-432d-863d-72720746da25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.110850 Aug 07 01:21:43.606793 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 586/1205] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:43 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:43.665088 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d3612096-319a-4747-b2f8-4e36b084096b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:43.668948 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d3612096-319a-4747-b2f8-4e36b084096b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:43.681505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d3612096-319a-4747-b2f8-4e36b084096b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:43.682421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d3612096-319a-4747-b2f8-4e36b084096b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:43.683210 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d3612096-319a-4747-b2f8-4e36b084096b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:43.684529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d3612096-319a-4747-b2f8-4e36b084096b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:43.685390 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d3612096-319a-4747-b2f8-4e36b084096b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:43.686166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d3612096-319a-4747-b2f8-4e36b084096b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:43.745500 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d3612096-319a-4747-b2f8-4e36b084096b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.088285 Aug 07 01:21:43.746586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 620/1206] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:43 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:44.629248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a08dd53d-3b55-43b8-b678-7642999e9fb2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:44.629966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a08dd53d-3b55-43b8-b678-7642999e9fb2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:44.637433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a08dd53d-3b55-43b8-b678-7642999e9fb2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:44.637931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a08dd53d-3b55-43b8-b678-7642999e9fb2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:44.638372 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a08dd53d-3b55-43b8-b678-7642999e9fb2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:44.639128 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a08dd53d-3b55-43b8-b678-7642999e9fb2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:44.639593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a08dd53d-3b55-43b8-b678-7642999e9fb2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:44.640033 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a08dd53d-3b55-43b8-b678-7642999e9fb2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:44.727639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a08dd53d-3b55-43b8-b678-7642999e9fb2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:44.739138 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a08dd53d-3b55-43b8-b678-7642999e9fb2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.114586 Aug 07 01:21:44.740027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 587/1207] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:44 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:44.761714 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-add38671-1200-47fd-b2d1-a9ab092c3889 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:44.764484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-add38671-1200-47fd-b2d1-a9ab092c3889 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:44.772563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-add38671-1200-47fd-b2d1-a9ab092c3889 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:44.773063 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-add38671-1200-47fd-b2d1-a9ab092c3889 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:44.773516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-add38671-1200-47fd-b2d1-a9ab092c3889 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:44.774276 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-add38671-1200-47fd-b2d1-a9ab092c3889 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:44.774721 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-add38671-1200-47fd-b2d1-a9ab092c3889 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:44.775152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-add38671-1200-47fd-b2d1-a9ab092c3889 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:44.810417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-add38671-1200-47fd-b2d1-a9ab092c3889 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052945 Aug 07 01:21:44.811016 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 621/1208] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:44 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:45.773671 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-96eda516-d13c-4d49-b727-de0899b9d5b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:45.774593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-96eda516-d13c-4d49-b727-de0899b9d5b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:45.782266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-96eda516-d13c-4d49-b727-de0899b9d5b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:45.782763 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-96eda516-d13c-4d49-b727-de0899b9d5b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:45.783208 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-96eda516-d13c-4d49-b727-de0899b9d5b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:45.784511 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-96eda516-d13c-4d49-b727-de0899b9d5b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:45.784973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-96eda516-d13c-4d49-b727-de0899b9d5b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:45.785401 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-96eda516-d13c-4d49-b727-de0899b9d5b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:45.831395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f37e29ee-9eb9-4013-b3a5-c2f07422e704 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:45.835622 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f37e29ee-9eb9-4013-b3a5-c2f07422e704 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:45.848098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f37e29ee-9eb9-4013-b3a5-c2f07422e704 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:45.850076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f37e29ee-9eb9-4013-b3a5-c2f07422e704 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:45.850076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f37e29ee-9eb9-4013-b3a5-c2f07422e704 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:45.857175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f37e29ee-9eb9-4013-b3a5-c2f07422e704 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:45.857565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f37e29ee-9eb9-4013-b3a5-c2f07422e704 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:45.858107 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f37e29ee-9eb9-4013-b3a5-c2f07422e704 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:45.860094 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-96eda516-d13c-4d49-b727-de0899b9d5b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:45.871311 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-96eda516-d13c-4d49-b727-de0899b9d5b3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.102200 Aug 07 01:21:45.872540 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 588/1209] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:45 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:45.913937 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f37e29ee-9eb9-4013-b3a5-c2f07422e704 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.091096 Aug 07 01:21:45.914956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 622/1210] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:45 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:46.900222 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-87e2a943-a86c-420b-a0c9-a14b674f430d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:46.901191 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-87e2a943-a86c-420b-a0c9-a14b674f430d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:46.907978 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-87e2a943-a86c-420b-a0c9-a14b674f430d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:46.908485 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-87e2a943-a86c-420b-a0c9-a14b674f430d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:46.909647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-87e2a943-a86c-420b-a0c9-a14b674f430d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:46.910440 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-87e2a943-a86c-420b-a0c9-a14b674f430d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:46.910989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-87e2a943-a86c-420b-a0c9-a14b674f430d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:46.911500 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-87e2a943-a86c-420b-a0c9-a14b674f430d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:46.916473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:21:46.928924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9544db6c-3613-4dc6-ae45-061572355e4a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:46.931589 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9544db6c-3613-4dc6-ae45-061572355e4a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:46.940157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9544db6c-3613-4dc6-ae45-061572355e4a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:46.940854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9544db6c-3613-4dc6-ae45-061572355e4a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:46.943044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9544db6c-3613-4dc6-ae45-061572355e4a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:46.944424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9544db6c-3613-4dc6-ae45-061572355e4a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:46.945048 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9544db6c-3613-4dc6-ae45-061572355e4a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:46.945662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9544db6c-3613-4dc6-ae45-061572355e4a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:46.985644 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9544db6c-3613-4dc6-ae45-061572355e4a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061067 Aug 07 01:21:46.986599 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 623/1211] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:46 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:46.999796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-87e2a943-a86c-420b-a0c9-a14b674f430d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:47.014031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-87e2a943-a86c-420b-a0c9-a14b674f430d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.117493 Aug 07 01:21:47.015118 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 589/1212] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:46 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:47.999475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-802c1b72-69cf-4bb1-9a51-162348c7a273 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:48.001619 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-802c1b72-69cf-4bb1-9a51-162348c7a273 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:48.009882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-802c1b72-69cf-4bb1-9a51-162348c7a273 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:48.010382 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-802c1b72-69cf-4bb1-9a51-162348c7a273 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:48.010825 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-802c1b72-69cf-4bb1-9a51-162348c7a273 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:48.011656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-802c1b72-69cf-4bb1-9a51-162348c7a273 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:48.012103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-802c1b72-69cf-4bb1-9a51-162348c7a273 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:48.012584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-802c1b72-69cf-4bb1-9a51-162348c7a273 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:48.044065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b93518b3-6a6d-4f12-822a-e77a44fdfcaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:48.045173 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b93518b3-6a6d-4f12-822a-e77a44fdfcaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:48.047446 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-802c1b72-69cf-4bb1-9a51-162348c7a273 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052376 Aug 07 01:21:48.048058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 624/1213] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:47 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:48.053791 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b93518b3-6a6d-4f12-822a-e77a44fdfcaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:48.054338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b93518b3-6a6d-4f12-822a-e77a44fdfcaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:48.054929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b93518b3-6a6d-4f12-822a-e77a44fdfcaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:48.055870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b93518b3-6a6d-4f12-822a-e77a44fdfcaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:48.056471 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b93518b3-6a6d-4f12-822a-e77a44fdfcaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:48.057162 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b93518b3-6a6d-4f12-822a-e77a44fdfcaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:48.137697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b93518b3-6a6d-4f12-822a-e77a44fdfcaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:48.150122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b93518b3-6a6d-4f12-822a-e77a44fdfcaf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.110784 Aug 07 01:21:48.151028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 590/1214] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:48 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:49.061363 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e3a6c0cb-188d-49a3-b826-492129c4362f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:49.063900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e3a6c0cb-188d-49a3-b826-492129c4362f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:49.071142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3a6c0cb-188d-49a3-b826-492129c4362f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:49.071651 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3a6c0cb-188d-49a3-b826-492129c4362f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:49.072129 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3a6c0cb-188d-49a3-b826-492129c4362f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:49.072904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3a6c0cb-188d-49a3-b826-492129c4362f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:49.073423 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3a6c0cb-188d-49a3-b826-492129c4362f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:49.073885 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3a6c0cb-188d-49a3-b826-492129c4362f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:49.111519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e3a6c0cb-188d-49a3-b826-492129c4362f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054301 Aug 07 01:21:49.112365 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 625/1215] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:49 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:49.171862 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1a1f4fc5-1eaa-4dab-b13d-a824a38e65aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:49.172515 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1a1f4fc5-1eaa-4dab-b13d-a824a38e65aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:49.179572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a1f4fc5-1eaa-4dab-b13d-a824a38e65aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:49.180117 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a1f4fc5-1eaa-4dab-b13d-a824a38e65aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:49.180647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a1f4fc5-1eaa-4dab-b13d-a824a38e65aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:49.181645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a1f4fc5-1eaa-4dab-b13d-a824a38e65aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:49.182206 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a1f4fc5-1eaa-4dab-b13d-a824a38e65aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:49.182688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a1f4fc5-1eaa-4dab-b13d-a824a38e65aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:49.267431 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1a1f4fc5-1eaa-4dab-b13d-a824a38e65aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:49.277019 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1a1f4fc5-1eaa-4dab-b13d-a824a38e65aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.109344 Aug 07 01:21:49.277506 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 591/1216] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:49 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:50.135396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a8903a01-28ee-45a6-b84a-e353217ace6f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:50.137674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a8903a01-28ee-45a6-b84a-e353217ace6f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:50.145676 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8903a01-28ee-45a6-b84a-e353217ace6f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:50.146017 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8903a01-28ee-45a6-b84a-e353217ace6f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:50.146460 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8903a01-28ee-45a6-b84a-e353217ace6f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:50.147208 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8903a01-28ee-45a6-b84a-e353217ace6f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:50.147677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8903a01-28ee-45a6-b84a-e353217ace6f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:50.148212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8903a01-28ee-45a6-b84a-e353217ace6f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:50.182923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a8903a01-28ee-45a6-b84a-e353217ace6f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051816 Aug 07 01:21:50.183539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 626/1217] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:50 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:50.303611 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f97d4594-3726-4e6d-ab57-60839b3aa9c8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:50.304704 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f97d4594-3726-4e6d-ab57-60839b3aa9c8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:50.311845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f97d4594-3726-4e6d-ab57-60839b3aa9c8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:50.312383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f97d4594-3726-4e6d-ab57-60839b3aa9c8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:50.312829 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f97d4594-3726-4e6d-ab57-60839b3aa9c8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:50.313549 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f97d4594-3726-4e6d-ab57-60839b3aa9c8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:50.314001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f97d4594-3726-4e6d-ab57-60839b3aa9c8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:50.314430 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f97d4594-3726-4e6d-ab57-60839b3aa9c8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:50.404731 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f97d4594-3726-4e6d-ab57-60839b3aa9c8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:50.420290 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f97d4594-3726-4e6d-ab57-60839b3aa9c8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.121876 Aug 07 01:21:50.421349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 592/1218] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:50 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:21:51.198895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-82613b0d-cad5-40bd-8121-376e4c6c6158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:51.202454 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-82613b0d-cad5-40bd-8121-376e4c6c6158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:51.216430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-82613b0d-cad5-40bd-8121-376e4c6c6158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:51.217662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-82613b0d-cad5-40bd-8121-376e4c6c6158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:51.218532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-82613b0d-cad5-40bd-8121-376e4c6c6158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:51.219875 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-82613b0d-cad5-40bd-8121-376e4c6c6158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:51.220753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-82613b0d-cad5-40bd-8121-376e4c6c6158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:51.221537 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-82613b0d-cad5-40bd-8121-376e4c6c6158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:51.284960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-82613b0d-cad5-40bd-8121-376e4c6c6158 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.091138 Aug 07 01:21:51.286292 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 627/1219] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:51 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:51.463207 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-eef1f860-602c-4044-a27f-db8696079e40 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:51.464492 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-eef1f860-602c-4044-a27f-db8696079e40 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:51.476565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eef1f860-602c-4044-a27f-db8696079e40 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:51.477441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eef1f860-602c-4044-a27f-db8696079e40 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:51.478302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eef1f860-602c-4044-a27f-db8696079e40 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:51.479825 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eef1f860-602c-4044-a27f-db8696079e40 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:51.480760 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eef1f860-602c-4044-a27f-db8696079e40 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:51.481496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eef1f860-602c-4044-a27f-db8696079e40 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:51.597282 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-eef1f860-602c-4044-a27f-db8696079e40 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:51.612843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-eef1f860-602c-4044-a27f-db8696079e40 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.157729 Aug 07 01:21:51.613916 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 593/1220] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:51 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:52.300983 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-98a9e4e4-27c4-4c5a-ad7b-ad56e18abc82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:52.303389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-98a9e4e4-27c4-4c5a-ad7b-ad56e18abc82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:52.311073 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98a9e4e4-27c4-4c5a-ad7b-ad56e18abc82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:52.311585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98a9e4e4-27c4-4c5a-ad7b-ad56e18abc82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:52.312038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98a9e4e4-27c4-4c5a-ad7b-ad56e18abc82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:52.313076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98a9e4e4-27c4-4c5a-ad7b-ad56e18abc82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:52.313612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98a9e4e4-27c4-4c5a-ad7b-ad56e18abc82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:52.314099 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98a9e4e4-27c4-4c5a-ad7b-ad56e18abc82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:52.353903 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-98a9e4e4-27c4-4c5a-ad7b-ad56e18abc82 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057687 Aug 07 01:21:52.354790 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 628/1221] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:52 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:21:52.650444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-31622bb3-8d26-4bc7-b189-febc35e0ce3d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:52.651332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-31622bb3-8d26-4bc7-b189-febc35e0ce3d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:52.660029 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-31622bb3-8d26-4bc7-b189-febc35e0ce3d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:52.660567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-31622bb3-8d26-4bc7-b189-febc35e0ce3d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:52.661027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-31622bb3-8d26-4bc7-b189-febc35e0ce3d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:52.661913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-31622bb3-8d26-4bc7-b189-febc35e0ce3d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:52.662570 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-31622bb3-8d26-4bc7-b189-febc35e0ce3d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:52.663041 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-31622bb3-8d26-4bc7-b189-febc35e0ce3d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:52.766860 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-31622bb3-8d26-4bc7-b189-febc35e0ce3d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:52.782381 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-31622bb3-8d26-4bc7-b189-febc35e0ce3d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.136029 Aug 07 01:21:52.783955 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 594/1222] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:52 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:53.375620 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3f91f954-d0bf-4ee9-88b5-b6a0b6cb32cc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:53.379286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3f91f954-d0bf-4ee9-88b5-b6a0b6cb32cc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:53.390270 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f91f954-d0bf-4ee9-88b5-b6a0b6cb32cc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:53.391094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f91f954-d0bf-4ee9-88b5-b6a0b6cb32cc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:53.391870 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f91f954-d0bf-4ee9-88b5-b6a0b6cb32cc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:53.393175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f91f954-d0bf-4ee9-88b5-b6a0b6cb32cc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:53.393956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f91f954-d0bf-4ee9-88b5-b6a0b6cb32cc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:53.394680 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f91f954-d0bf-4ee9-88b5-b6a0b6cb32cc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:53.463224 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3f91f954-d0bf-4ee9-88b5-b6a0b6cb32cc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.094328 Aug 07 01:21:53.464376 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 629/1223] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:53 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:53.819545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3a9002c1-2db7-454f-becb-188fbc55d2bd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:53.820417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3a9002c1-2db7-454f-becb-188fbc55d2bd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:53.832168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3a9002c1-2db7-454f-becb-188fbc55d2bd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:53.833073 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3a9002c1-2db7-454f-becb-188fbc55d2bd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:53.834125 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3a9002c1-2db7-454f-becb-188fbc55d2bd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:53.835504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3a9002c1-2db7-454f-becb-188fbc55d2bd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:53.836412 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3a9002c1-2db7-454f-becb-188fbc55d2bd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:53.837197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3a9002c1-2db7-454f-becb-188fbc55d2bd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:53.923758 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3a9002c1-2db7-454f-becb-188fbc55d2bd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:53.937791 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3a9002c1-2db7-454f-becb-188fbc55d2bd tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.123627 Aug 07 01:21:53.938652 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 595/1224] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:53 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:54.479738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-484ede16-1a3e-446a-8d32-92e67964f15c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:54.482100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-484ede16-1a3e-446a-8d32-92e67964f15c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:54.489664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-484ede16-1a3e-446a-8d32-92e67964f15c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:54.490164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-484ede16-1a3e-446a-8d32-92e67964f15c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:54.490625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-484ede16-1a3e-446a-8d32-92e67964f15c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:54.491366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-484ede16-1a3e-446a-8d32-92e67964f15c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:54.491830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-484ede16-1a3e-446a-8d32-92e67964f15c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:54.492328 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-484ede16-1a3e-446a-8d32-92e67964f15c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:54.512992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:21:54.532541 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-484ede16-1a3e-446a-8d32-92e67964f15c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056798 Aug 07 01:21:54.533125 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 630/1225] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:54 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:54.972934 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-358f9318-6e1a-4d35-851b-bfbb49aabece tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:54.973530 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-358f9318-6e1a-4d35-851b-bfbb49aabece tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:54.981049 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-358f9318-6e1a-4d35-851b-bfbb49aabece tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:54.981546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-358f9318-6e1a-4d35-851b-bfbb49aabece tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:54.981974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-358f9318-6e1a-4d35-851b-bfbb49aabece tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:54.982709 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-358f9318-6e1a-4d35-851b-bfbb49aabece tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:54.983145 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-358f9318-6e1a-4d35-851b-bfbb49aabece tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:54.983571 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-358f9318-6e1a-4d35-851b-bfbb49aabece tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:55.058140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-358f9318-6e1a-4d35-851b-bfbb49aabece tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:55.071961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-358f9318-6e1a-4d35-851b-bfbb49aabece tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.102980 Aug 07 01:21:55.073038 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 596/1226] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:54 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:55.546273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ebe67807-5844-4e2d-96c6-e021e8492fc0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:55.548649 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ebe67807-5844-4e2d-96c6-e021e8492fc0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:55.556603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ebe67807-5844-4e2d-96c6-e021e8492fc0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:55.557091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ebe67807-5844-4e2d-96c6-e021e8492fc0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:55.557536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ebe67807-5844-4e2d-96c6-e021e8492fc0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:55.558272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ebe67807-5844-4e2d-96c6-e021e8492fc0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:55.558723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ebe67807-5844-4e2d-96c6-e021e8492fc0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:55.559196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ebe67807-5844-4e2d-96c6-e021e8492fc0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:55.594513 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ebe67807-5844-4e2d-96c6-e021e8492fc0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052769 Aug 07 01:21:55.595173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 631/1227] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:55 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:56.093961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8ac8311f-2a79-48b6-ba80-9a4f43e717ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:56.094628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8ac8311f-2a79-48b6-ba80-9a4f43e717ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:56.101859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ac8311f-2a79-48b6-ba80-9a4f43e717ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:56.102344 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ac8311f-2a79-48b6-ba80-9a4f43e717ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:56.102783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ac8311f-2a79-48b6-ba80-9a4f43e717ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:56.103496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ac8311f-2a79-48b6-ba80-9a4f43e717ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:56.103948 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ac8311f-2a79-48b6-ba80-9a4f43e717ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:56.104432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ac8311f-2a79-48b6-ba80-9a4f43e717ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:56.179660 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8ac8311f-2a79-48b6-ba80-9a4f43e717ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:56.190702 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8ac8311f-2a79-48b6-ba80-9a4f43e717ef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.099617 Aug 07 01:21:56.191375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 597/1228] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:56 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:56.609210 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-01f1e490-9d22-4f24-a5ea-9ca3c742dcb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:56.612163 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-01f1e490-9d22-4f24-a5ea-9ca3c742dcb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:56.621042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-01f1e490-9d22-4f24-a5ea-9ca3c742dcb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:56.621633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-01f1e490-9d22-4f24-a5ea-9ca3c742dcb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:56.622180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-01f1e490-9d22-4f24-a5ea-9ca3c742dcb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:56.623114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-01f1e490-9d22-4f24-a5ea-9ca3c742dcb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:56.623635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-01f1e490-9d22-4f24-a5ea-9ca3c742dcb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:56.624185 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-01f1e490-9d22-4f24-a5ea-9ca3c742dcb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:56.662436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-01f1e490-9d22-4f24-a5ea-9ca3c742dcb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058357 Aug 07 01:21:56.663081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 632/1229] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:56 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:57.214886 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fb86f2a4-23b2-45a8-af80-f42b2163072a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:57.215880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fb86f2a4-23b2-45a8-af80-f42b2163072a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:57.223094 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb86f2a4-23b2-45a8-af80-f42b2163072a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:57.223725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb86f2a4-23b2-45a8-af80-f42b2163072a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:57.224407 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb86f2a4-23b2-45a8-af80-f42b2163072a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:57.225335 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb86f2a4-23b2-45a8-af80-f42b2163072a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:57.225975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb86f2a4-23b2-45a8-af80-f42b2163072a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:57.226496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb86f2a4-23b2-45a8-af80-f42b2163072a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:57.318713 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fb86f2a4-23b2-45a8-af80-f42b2163072a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:57.328335 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fb86f2a4-23b2-45a8-af80-f42b2163072a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.117600 Aug 07 01:21:57.329029 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 598/1230] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:57 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:57.676631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-60210bac-988f-45ca-820f-9967c3890214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:57.679075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-60210bac-988f-45ca-820f-9967c3890214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:57.686974 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60210bac-988f-45ca-820f-9967c3890214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:57.687525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60210bac-988f-45ca-820f-9967c3890214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:57.688009 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60210bac-988f-45ca-820f-9967c3890214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:57.688961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60210bac-988f-45ca-820f-9967c3890214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:57.689595 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60210bac-988f-45ca-820f-9967c3890214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:57.689927 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-60210bac-988f-45ca-820f-9967c3890214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:57.728590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-60210bac-988f-45ca-820f-9967c3890214 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057065 Aug 07 01:21:57.729171 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 633/1231] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:57 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:58.368138 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ad5068f7-05b3-4303-abc1-a4ab469207bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:58.369149 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ad5068f7-05b3-4303-abc1-a4ab469207bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:58.378525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ad5068f7-05b3-4303-abc1-a4ab469207bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:58.379180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ad5068f7-05b3-4303-abc1-a4ab469207bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:58.379732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ad5068f7-05b3-4303-abc1-a4ab469207bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:58.380693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ad5068f7-05b3-4303-abc1-a4ab469207bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:58.381270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ad5068f7-05b3-4303-abc1-a4ab469207bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:58.381806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ad5068f7-05b3-4303-abc1-a4ab469207bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:58.459237 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ad5068f7-05b3-4303-abc1-a4ab469207bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:58.472929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ad5068f7-05b3-4303-abc1-a4ab469207bb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.115157 Aug 07 01:21:58.473598 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 599/1232] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:58 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:58.747551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8c8f9f33-a910-40e7-8de8-97342583138c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:58.750249 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8c8f9f33-a910-40e7-8de8-97342583138c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:58.758441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c8f9f33-a910-40e7-8de8-97342583138c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:58.759008 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c8f9f33-a910-40e7-8de8-97342583138c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:58.759572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c8f9f33-a910-40e7-8de8-97342583138c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:58.760431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c8f9f33-a910-40e7-8de8-97342583138c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:58.761001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c8f9f33-a910-40e7-8de8-97342583138c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:58.761484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c8f9f33-a910-40e7-8de8-97342583138c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:58.796478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8c8f9f33-a910-40e7-8de8-97342583138c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057103 Aug 07 01:21:58.797072 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 634/1233] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:58 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:21:59.506926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7affc98f-eded-4aa0-a677-fb6fa881697f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:59.507746 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7affc98f-eded-4aa0-a677-fb6fa881697f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:59.518846 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7affc98f-eded-4aa0-a677-fb6fa881697f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:59.519728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7affc98f-eded-4aa0-a677-fb6fa881697f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:59.520565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7affc98f-eded-4aa0-a677-fb6fa881697f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:59.521891 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7affc98f-eded-4aa0-a677-fb6fa881697f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:59.522733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7affc98f-eded-4aa0-a677-fb6fa881697f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:59.523536 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7affc98f-eded-4aa0-a677-fb6fa881697f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:59.630041 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7affc98f-eded-4aa0-a677-fb6fa881697f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:21:59.639867 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7affc98f-eded-4aa0-a677-fb6fa881697f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.138086 Aug 07 01:21:59.640650 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 600/1234] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:21:59 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:21:59.813191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a74a5646-4ab4-4c04-b52b-6527a75989f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:21:59.817614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a74a5646-4ab4-4c04-b52b-6527a75989f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:21:59.829356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a74a5646-4ab4-4c04-b52b-6527a75989f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:59.830757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a74a5646-4ab4-4c04-b52b-6527a75989f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:59.831547 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a74a5646-4ab4-4c04-b52b-6527a75989f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:59.832770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a74a5646-4ab4-4c04-b52b-6527a75989f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:21:59.834281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a74a5646-4ab4-4c04-b52b-6527a75989f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:21:59.835123 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a74a5646-4ab4-4c04-b52b-6527a75989f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:21:59.899431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a74a5646-4ab4-4c04-b52b-6527a75989f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.093087 Aug 07 01:21:59.900579 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 635/1235] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:21:59 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:00.681384 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bceac7c9-78c5-41bb-ad64-d153ca937631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:00.682454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bceac7c9-78c5-41bb-ad64-d153ca937631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:00.693851 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bceac7c9-78c5-41bb-ad64-d153ca937631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:00.694369 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bceac7c9-78c5-41bb-ad64-d153ca937631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:00.694877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bceac7c9-78c5-41bb-ad64-d153ca937631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:00.696059 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bceac7c9-78c5-41bb-ad64-d153ca937631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:00.697057 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bceac7c9-78c5-41bb-ad64-d153ca937631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:00.697771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bceac7c9-78c5-41bb-ad64-d153ca937631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:00.777259 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bceac7c9-78c5-41bb-ad64-d153ca937631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:00.785852 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bceac7c9-78c5-41bb-ad64-d153ca937631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.113579 Aug 07 01:22:00.786732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 601/1236] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:00 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:00.916584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-851254eb-079b-4d7c-bffb-72ee9e5b8f78 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:00.918806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-851254eb-079b-4d7c-bffb-72ee9e5b8f78 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:00.928324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-851254eb-079b-4d7c-bffb-72ee9e5b8f78 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:00.928825 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-851254eb-079b-4d7c-bffb-72ee9e5b8f78 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:00.929272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-851254eb-079b-4d7c-bffb-72ee9e5b8f78 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:00.930449 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-851254eb-079b-4d7c-bffb-72ee9e5b8f78 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:00.930969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-851254eb-079b-4d7c-bffb-72ee9e5b8f78 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:00.931579 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-851254eb-079b-4d7c-bffb-72ee9e5b8f78 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:00.971082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-851254eb-079b-4d7c-bffb-72ee9e5b8f78 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058872 Aug 07 01:22:00.971718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 636/1237] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:00 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:01.810831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9469ebbe-cae9-4307-bd95-4512c70bc1c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:01.811957 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9469ebbe-cae9-4307-bd95-4512c70bc1c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:01.823788 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9469ebbe-cae9-4307-bd95-4512c70bc1c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:01.824956 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9469ebbe-cae9-4307-bd95-4512c70bc1c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:01.825738 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9469ebbe-cae9-4307-bd95-4512c70bc1c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:01.826973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9469ebbe-cae9-4307-bd95-4512c70bc1c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:01.827787 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9469ebbe-cae9-4307-bd95-4512c70bc1c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:01.828546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9469ebbe-cae9-4307-bd95-4512c70bc1c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:01.931554 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9469ebbe-cae9-4307-bd95-4512c70bc1c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:01.946910 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9469ebbe-cae9-4307-bd95-4512c70bc1c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.142931 Aug 07 01:22:01.948037 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 602/1238] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:01 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:01.987969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7f3c1c72-44c8-4de7-9315-cb6fd1a8a684 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:01.991618 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7f3c1c72-44c8-4de7-9315-cb6fd1a8a684 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:02.004142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7f3c1c72-44c8-4de7-9315-cb6fd1a8a684 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:02.005752 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7f3c1c72-44c8-4de7-9315-cb6fd1a8a684 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:02.005752 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7f3c1c72-44c8-4de7-9315-cb6fd1a8a684 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:02.007728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7f3c1c72-44c8-4de7-9315-cb6fd1a8a684 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:02.008083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7f3c1c72-44c8-4de7-9315-cb6fd1a8a684 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:02.008721 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7f3c1c72-44c8-4de7-9315-cb6fd1a8a684 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:02.065315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7f3c1c72-44c8-4de7-9315-cb6fd1a8a684 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.084154 Aug 07 01:22:02.066286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 637/1239] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:01 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:02.988032 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4d6cd7e3-e12d-46dc-b65f-e41aa448e05d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:02.989148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4d6cd7e3-e12d-46dc-b65f-e41aa448e05d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:02.997933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d6cd7e3-e12d-46dc-b65f-e41aa448e05d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:02.998442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d6cd7e3-e12d-46dc-b65f-e41aa448e05d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:02.998887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d6cd7e3-e12d-46dc-b65f-e41aa448e05d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:02.999628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d6cd7e3-e12d-46dc-b65f-e41aa448e05d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:03.000150 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d6cd7e3-e12d-46dc-b65f-e41aa448e05d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:03.000788 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d6cd7e3-e12d-46dc-b65f-e41aa448e05d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:03.082690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-724f53cd-56b6-4b84-8fe5-d11215de55e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:03.085692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4d6cd7e3-e12d-46dc-b65f-e41aa448e05d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:03.087014 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-724f53cd-56b6-4b84-8fe5-d11215de55e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:03.091421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=56 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:22:03.098596 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4d6cd7e3-e12d-46dc-b65f-e41aa448e05d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.118555 Aug 07 01:22:03.099432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 603/1240] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:02 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:03.102903 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-724f53cd-56b6-4b84-8fe5-d11215de55e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:03.103842 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-724f53cd-56b6-4b84-8fe5-d11215de55e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:03.104735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-724f53cd-56b6-4b84-8fe5-d11215de55e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:03.106094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-724f53cd-56b6-4b84-8fe5-d11215de55e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:03.106957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-724f53cd-56b6-4b84-8fe5-d11215de55e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:03.107747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-724f53cd-56b6-4b84-8fe5-d11215de55e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:03.168529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-724f53cd-56b6-4b84-8fe5-d11215de55e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.091155 Aug 07 01:22:03.169444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 638/1241] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:03 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:04.127109 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-38e2d803-ac93-4de9-a9b4-8911cc353202 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:04.128147 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-38e2d803-ac93-4de9-a9b4-8911cc353202 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:04.139931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-38e2d803-ac93-4de9-a9b4-8911cc353202 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:04.140818 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-38e2d803-ac93-4de9-a9b4-8911cc353202 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:04.141614 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-38e2d803-ac93-4de9-a9b4-8911cc353202 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:04.142913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-38e2d803-ac93-4de9-a9b4-8911cc353202 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:04.143761 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-38e2d803-ac93-4de9-a9b4-8911cc353202 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:04.144612 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-38e2d803-ac93-4de9-a9b4-8911cc353202 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:04.189947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ccc0f9af-7afa-422b-9bbe-911a48d347d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:04.194497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ccc0f9af-7afa-422b-9bbe-911a48d347d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:04.206873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccc0f9af-7afa-422b-9bbe-911a48d347d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:04.207786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccc0f9af-7afa-422b-9bbe-911a48d347d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:04.209187 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccc0f9af-7afa-422b-9bbe-911a48d347d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:04.210625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccc0f9af-7afa-422b-9bbe-911a48d347d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:04.211489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccc0f9af-7afa-422b-9bbe-911a48d347d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:04.212345 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ccc0f9af-7afa-422b-9bbe-911a48d347d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:04.266391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-38e2d803-ac93-4de9-a9b4-8911cc353202 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:04.269702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ccc0f9af-7afa-422b-9bbe-911a48d347d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.087874 Aug 07 01:22:04.271122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 639/1242] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:04 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:04.281466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-38e2d803-ac93-4de9-a9b4-8911cc353202 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.162861 Aug 07 01:22:04.282642 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 604/1243] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:04 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:05.290592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2007ec17-f057-4161-b808-fcc8c45eb0af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:05.294081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2007ec17-f057-4161-b808-fcc8c45eb0af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:05.306386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2007ec17-f057-4161-b808-fcc8c45eb0af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:05.307109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2007ec17-f057-4161-b808-fcc8c45eb0af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:05.307794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2007ec17-f057-4161-b808-fcc8c45eb0af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:05.309058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2007ec17-f057-4161-b808-fcc8c45eb0af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:05.311266 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2007ec17-f057-4161-b808-fcc8c45eb0af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:05.312096 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2007ec17-f057-4161-b808-fcc8c45eb0af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:05.322832 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-eeae0bbf-9077-42ca-981b-ee0f1b3ab817 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:05.322832 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-eeae0bbf-9077-42ca-981b-ee0f1b3ab817 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:05.326041 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=54 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:22:05.335833 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eeae0bbf-9077-42ca-981b-ee0f1b3ab817 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:05.336632 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eeae0bbf-9077-42ca-981b-ee0f1b3ab817 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:05.337373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eeae0bbf-9077-42ca-981b-ee0f1b3ab817 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:05.338557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eeae0bbf-9077-42ca-981b-ee0f1b3ab817 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:05.339398 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eeae0bbf-9077-42ca-981b-ee0f1b3ab817 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:05.340087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eeae0bbf-9077-42ca-981b-ee0f1b3ab817 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:05.375576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2007ec17-f057-4161-b808-fcc8c45eb0af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.092485 Aug 07 01:22:05.376821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 640/1244] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:05 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:05.454728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-eeae0bbf-9077-42ca-981b-ee0f1b3ab817 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:05.460841 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:22:05.470619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-eeae0bbf-9077-42ca-981b-ee0f1b3ab817 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.156677 Aug 07 01:22:05.471798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 605/1245] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:05 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:06.395067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-71a58a8d-04dc-4d66-8ace-d75e21b13399 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:06.399171 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-71a58a8d-04dc-4d66-8ace-d75e21b13399 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:06.411513 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71a58a8d-04dc-4d66-8ace-d75e21b13399 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:06.412402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71a58a8d-04dc-4d66-8ace-d75e21b13399 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:06.413216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71a58a8d-04dc-4d66-8ace-d75e21b13399 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:06.414552 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71a58a8d-04dc-4d66-8ace-d75e21b13399 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:06.415388 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71a58a8d-04dc-4d66-8ace-d75e21b13399 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:06.416198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71a58a8d-04dc-4d66-8ace-d75e21b13399 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:06.478771 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-71a58a8d-04dc-4d66-8ace-d75e21b13399 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.089961 Aug 07 01:22:06.479831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 641/1246] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:06 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:06.506167 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2ccb4b81-d950-41b7-a2fd-a27a7b02864f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:06.508528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2ccb4b81-d950-41b7-a2fd-a27a7b02864f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:06.514173 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2ccb4b81-d950-41b7-a2fd-a27a7b02864f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:06.514735 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2ccb4b81-d950-41b7-a2fd-a27a7b02864f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:06.515246 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2ccb4b81-d950-41b7-a2fd-a27a7b02864f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:06.516043 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2ccb4b81-d950-41b7-a2fd-a27a7b02864f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:06.516594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2ccb4b81-d950-41b7-a2fd-a27a7b02864f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:06.517116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2ccb4b81-d950-41b7-a2fd-a27a7b02864f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:06.609130 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2ccb4b81-d950-41b7-a2fd-a27a7b02864f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:06.622831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2ccb4b81-d950-41b7-a2fd-a27a7b02864f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.120921 Aug 07 01:22:06.623882 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 606/1247] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:06 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:07.499047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ef5bd6ae-8de7-48db-89b9-6d261da10ae2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:07.503447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ef5bd6ae-8de7-48db-89b9-6d261da10ae2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:07.514589 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef5bd6ae-8de7-48db-89b9-6d261da10ae2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:07.515392 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef5bd6ae-8de7-48db-89b9-6d261da10ae2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:07.516166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef5bd6ae-8de7-48db-89b9-6d261da10ae2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:07.517452 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef5bd6ae-8de7-48db-89b9-6d261da10ae2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:07.518201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef5bd6ae-8de7-48db-89b9-6d261da10ae2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:07.518979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef5bd6ae-8de7-48db-89b9-6d261da10ae2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:07.575608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ef5bd6ae-8de7-48db-89b9-6d261da10ae2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.083390 Aug 07 01:22:07.576562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 642/1248] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:07 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:07.656316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6039ae02-bdf8-467b-9299-278ac32511a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:07.657437 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6039ae02-bdf8-467b-9299-278ac32511a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:07.670649 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6039ae02-bdf8-467b-9299-278ac32511a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:07.671537 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6039ae02-bdf8-467b-9299-278ac32511a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:07.672381 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6039ae02-bdf8-467b-9299-278ac32511a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:07.673723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6039ae02-bdf8-467b-9299-278ac32511a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:07.674547 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6039ae02-bdf8-467b-9299-278ac32511a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:07.675330 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6039ae02-bdf8-467b-9299-278ac32511a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:07.779699 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6039ae02-bdf8-467b-9299-278ac32511a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:07.793620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6039ae02-bdf8-467b-9299-278ac32511a1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.144670 Aug 07 01:22:07.794231 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 607/1249] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:07 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:08.598126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7144c50b-dcab-461e-a37b-1911ced949e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:08.602033 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7144c50b-dcab-461e-a37b-1911ced949e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:08.614884 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7144c50b-dcab-461e-a37b-1911ced949e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:08.615800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7144c50b-dcab-461e-a37b-1911ced949e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:08.616648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7144c50b-dcab-461e-a37b-1911ced949e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:08.617991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7144c50b-dcab-461e-a37b-1911ced949e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:08.618816 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7144c50b-dcab-461e-a37b-1911ced949e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:08.619577 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7144c50b-dcab-461e-a37b-1911ced949e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:08.676883 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7144c50b-dcab-461e-a37b-1911ced949e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.086771 Aug 07 01:22:08.677836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 643/1250] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:08 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:08.828689 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6b7739b1-b862-4ac9-ac23-e9b243162cbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:08.830477 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6b7739b1-b862-4ac9-ac23-e9b243162cbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:08.842274 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b7739b1-b862-4ac9-ac23-e9b243162cbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:08.843082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b7739b1-b862-4ac9-ac23-e9b243162cbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:08.843780 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b7739b1-b862-4ac9-ac23-e9b243162cbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:08.844855 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b7739b1-b862-4ac9-ac23-e9b243162cbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:08.845817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b7739b1-b862-4ac9-ac23-e9b243162cbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:08.846586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b7739b1-b862-4ac9-ac23-e9b243162cbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:08.964661 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6b7739b1-b862-4ac9-ac23-e9b243162cbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:08.980352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6b7739b1-b862-4ac9-ac23-e9b243162cbf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.157796 Aug 07 01:22:08.981398 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 608/1251] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:08 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:09.698724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7d330009-b9db-49cf-bc21-487f64122c0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:09.699887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7d330009-b9db-49cf-bc21-487f64122c0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:09.708822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d330009-b9db-49cf-bc21-487f64122c0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:09.709330 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d330009-b9db-49cf-bc21-487f64122c0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:09.709781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d330009-b9db-49cf-bc21-487f64122c0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:09.710615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d330009-b9db-49cf-bc21-487f64122c0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:09.711445 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d330009-b9db-49cf-bc21-487f64122c0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:09.712153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d330009-b9db-49cf-bc21-487f64122c0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:09.772710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7d330009-b9db-49cf-bc21-487f64122c0d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.081735 Aug 07 01:22:09.773659 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 644/1252] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:09 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:10.023762 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-dc4967d9-a9ae-4f9c-b849-f45cc99822cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:10.024732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-dc4967d9-a9ae-4f9c-b849-f45cc99822cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:10.040753 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc4967d9-a9ae-4f9c-b849-f45cc99822cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:10.041527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc4967d9-a9ae-4f9c-b849-f45cc99822cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:10.042339 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc4967d9-a9ae-4f9c-b849-f45cc99822cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:10.043665 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc4967d9-a9ae-4f9c-b849-f45cc99822cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:10.044437 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc4967d9-a9ae-4f9c-b849-f45cc99822cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:10.045181 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc4967d9-a9ae-4f9c-b849-f45cc99822cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:10.163145 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dc4967d9-a9ae-4f9c-b849-f45cc99822cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:10.176381 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-dc4967d9-a9ae-4f9c-b849-f45cc99822cb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.159258 Aug 07 01:22:10.177342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 609/1253] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:10 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:10.793702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c65c7714-9ab4-42d2-a8de-637606415198 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:10.797692 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c65c7714-9ab4-42d2-a8de-637606415198 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:10.809932 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c65c7714-9ab4-42d2-a8de-637606415198 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:10.810858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c65c7714-9ab4-42d2-a8de-637606415198 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:10.811611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c65c7714-9ab4-42d2-a8de-637606415198 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:10.813013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c65c7714-9ab4-42d2-a8de-637606415198 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:10.813896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c65c7714-9ab4-42d2-a8de-637606415198 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:10.815244 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c65c7714-9ab4-42d2-a8de-637606415198 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:10.877042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c65c7714-9ab4-42d2-a8de-637606415198 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.092030 Aug 07 01:22:10.878086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 645/1254] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:10 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:11.215621 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ca38fee2-c91a-443e-bb2a-c7c72aab62ac tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:11.216545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ca38fee2-c91a-443e-bb2a-c7c72aab62ac tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:11.223836 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca38fee2-c91a-443e-bb2a-c7c72aab62ac tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:11.224474 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca38fee2-c91a-443e-bb2a-c7c72aab62ac tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:11.224915 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca38fee2-c91a-443e-bb2a-c7c72aab62ac tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:11.225647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca38fee2-c91a-443e-bb2a-c7c72aab62ac tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:11.226098 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca38fee2-c91a-443e-bb2a-c7c72aab62ac tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:11.226563 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca38fee2-c91a-443e-bb2a-c7c72aab62ac tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:11.310782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ca38fee2-c91a-443e-bb2a-c7c72aab62ac tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:11.325784 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ca38fee2-c91a-443e-bb2a-c7c72aab62ac tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.117341 Aug 07 01:22:11.326843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 610/1255] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:11 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:11.893897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0849f141-7e02-4ada-9b06-4fe6bfa0ab40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:11.896890 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0849f141-7e02-4ada-9b06-4fe6bfa0ab40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:11.904173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0849f141-7e02-4ada-9b06-4fe6bfa0ab40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:11.904709 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0849f141-7e02-4ada-9b06-4fe6bfa0ab40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:11.905154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0849f141-7e02-4ada-9b06-4fe6bfa0ab40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:11.905896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0849f141-7e02-4ada-9b06-4fe6bfa0ab40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:11.906358 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0849f141-7e02-4ada-9b06-4fe6bfa0ab40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:11.906788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0849f141-7e02-4ada-9b06-4fe6bfa0ab40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:11.949633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0849f141-7e02-4ada-9b06-4fe6bfa0ab40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060333 Aug 07 01:22:11.950239 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 646/1256] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:11 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:12.364415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-eb6d84e0-12a3-4953-8232-31bb0c89c193 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:12.365614 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-eb6d84e0-12a3-4953-8232-31bb0c89c193 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:12.377969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb6d84e0-12a3-4953-8232-31bb0c89c193 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:12.378824 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb6d84e0-12a3-4953-8232-31bb0c89c193 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:12.379596 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb6d84e0-12a3-4953-8232-31bb0c89c193 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:12.380941 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb6d84e0-12a3-4953-8232-31bb0c89c193 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:12.381782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb6d84e0-12a3-4953-8232-31bb0c89c193 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:12.382573 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb6d84e0-12a3-4953-8232-31bb0c89c193 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:12.488787 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-eb6d84e0-12a3-4953-8232-31bb0c89c193 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:12.502325 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-eb6d84e0-12a3-4953-8232-31bb0c89c193 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.145677 Aug 07 01:22:12.503393 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 611/1257] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:12 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:12.968641 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-af62a5c9-1138-473b-93de-6e6edf036a8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:12.973070 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-af62a5c9-1138-473b-93de-6e6edf036a8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:12.984125 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af62a5c9-1138-473b-93de-6e6edf036a8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:12.985019 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af62a5c9-1138-473b-93de-6e6edf036a8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:12.985993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af62a5c9-1138-473b-93de-6e6edf036a8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:12.987322 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af62a5c9-1138-473b-93de-6e6edf036a8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:12.988186 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af62a5c9-1138-473b-93de-6e6edf036a8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:12.989011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af62a5c9-1138-473b-93de-6e6edf036a8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:12.996811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:22:13.049656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-af62a5c9-1138-473b-93de-6e6edf036a8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.088062 Aug 07 01:22:13.050673 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 647/1258] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:12 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:13.538321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3f97e702-5c07-4757-9e62-a5d169e46390 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:13.539534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3f97e702-5c07-4757-9e62-a5d169e46390 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:13.552671 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3f97e702-5c07-4757-9e62-a5d169e46390 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:13.553568 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3f97e702-5c07-4757-9e62-a5d169e46390 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:13.554391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3f97e702-5c07-4757-9e62-a5d169e46390 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:13.555756 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3f97e702-5c07-4757-9e62-a5d169e46390 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:13.556665 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3f97e702-5c07-4757-9e62-a5d169e46390 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:13.557450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3f97e702-5c07-4757-9e62-a5d169e46390 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:13.658738 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3f97e702-5c07-4757-9e62-a5d169e46390 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:13.669659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3f97e702-5c07-4757-9e62-a5d169e46390 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.139013 Aug 07 01:22:13.670554 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 612/1259] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:13 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:14.067645 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f849f39b-7ee8-4f7a-9b68-f55c3d98d47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:14.070522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f849f39b-7ee8-4f7a-9b68-f55c3d98d47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:14.081953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f849f39b-7ee8-4f7a-9b68-f55c3d98d47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:14.082865 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f849f39b-7ee8-4f7a-9b68-f55c3d98d47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:14.083636 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f849f39b-7ee8-4f7a-9b68-f55c3d98d47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:14.085058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f849f39b-7ee8-4f7a-9b68-f55c3d98d47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:14.085898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f849f39b-7ee8-4f7a-9b68-f55c3d98d47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:14.086703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f849f39b-7ee8-4f7a-9b68-f55c3d98d47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:14.129448 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f849f39b-7ee8-4f7a-9b68-f55c3d98d47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067219 Aug 07 01:22:14.130159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 648/1260] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:14 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:14.707667 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-10a92e9e-aaef-4091-97b8-064ec4ce505d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:14.708350 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-10a92e9e-aaef-4091-97b8-064ec4ce505d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:14.716954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10a92e9e-aaef-4091-97b8-064ec4ce505d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:14.717581 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10a92e9e-aaef-4091-97b8-064ec4ce505d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:14.718095 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10a92e9e-aaef-4091-97b8-064ec4ce505d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:14.719113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10a92e9e-aaef-4091-97b8-064ec4ce505d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:14.719651 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10a92e9e-aaef-4091-97b8-064ec4ce505d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:14.720779 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-10a92e9e-aaef-4091-97b8-064ec4ce505d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:14.817387 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-10a92e9e-aaef-4091-97b8-064ec4ce505d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:14.832888 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-10a92e9e-aaef-4091-97b8-064ec4ce505d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.128917 Aug 07 01:22:14.833946 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 613/1261] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:14 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:22:15.321793 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3c826fb3-1ca0-4ac5-84ed-25723af285d5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:15.326316 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3c826fb3-1ca0-4ac5-84ed-25723af285d5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:15.338983 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c826fb3-1ca0-4ac5-84ed-25723af285d5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:15.339790 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c826fb3-1ca0-4ac5-84ed-25723af285d5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:15.340699 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c826fb3-1ca0-4ac5-84ed-25723af285d5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:15.342071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c826fb3-1ca0-4ac5-84ed-25723af285d5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:15.342943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c826fb3-1ca0-4ac5-84ed-25723af285d5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:15.343767 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c826fb3-1ca0-4ac5-84ed-25723af285d5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:15.405017 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3c826fb3-1ca0-4ac5-84ed-25723af285d5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.264724 Aug 07 01:22:15.406072 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 649/1262] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:15 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:15.871089 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a9a9f87c-b855-418e-bdfd-c25144a0f8b4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:15.872146 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a9a9f87c-b855-418e-bdfd-c25144a0f8b4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:15.884609 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a9a9f87c-b855-418e-bdfd-c25144a0f8b4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:15.885465 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a9a9f87c-b855-418e-bdfd-c25144a0f8b4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:15.886268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a9a9f87c-b855-418e-bdfd-c25144a0f8b4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:15.887626 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a9a9f87c-b855-418e-bdfd-c25144a0f8b4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:15.888497 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a9a9f87c-b855-418e-bdfd-c25144a0f8b4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:15.889271 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a9a9f87c-b855-418e-bdfd-c25144a0f8b4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:16.012032 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a9a9f87c-b855-418e-bdfd-c25144a0f8b4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:16.027716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a9a9f87c-b855-418e-bdfd-c25144a0f8b4 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.163896 Aug 07 01:22:16.028934 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 614/1263] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:15 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:16.426395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3247f1a7-1c33-4322-a684-58ca8e7b4904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:16.430381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3247f1a7-1c33-4322-a684-58ca8e7b4904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:16.443110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3247f1a7-1c33-4322-a684-58ca8e7b4904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:16.444067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3247f1a7-1c33-4322-a684-58ca8e7b4904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:16.444952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3247f1a7-1c33-4322-a684-58ca8e7b4904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:16.446325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3247f1a7-1c33-4322-a684-58ca8e7b4904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:16.447178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3247f1a7-1c33-4322-a684-58ca8e7b4904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:16.447983 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3247f1a7-1c33-4322-a684-58ca8e7b4904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:16.517765 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3247f1a7-1c33-4322-a684-58ca8e7b4904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.098363 Aug 07 01:22:16.518840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 650/1264] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:16 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:17.070800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-362f4024-865a-4c5b-87b2-117d45025266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:17.072048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-362f4024-865a-4c5b-87b2-117d45025266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:17.085401 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-362f4024-865a-4c5b-87b2-117d45025266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:17.086289 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-362f4024-865a-4c5b-87b2-117d45025266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:17.087154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-362f4024-865a-4c5b-87b2-117d45025266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:17.088589 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-362f4024-865a-4c5b-87b2-117d45025266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:17.089492 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-362f4024-865a-4c5b-87b2-117d45025266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:17.090257 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-362f4024-865a-4c5b-87b2-117d45025266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:17.205823 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-362f4024-865a-4c5b-87b2-117d45025266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:17.216127 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-362f4024-865a-4c5b-87b2-117d45025266 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.153628 Aug 07 01:22:17.216788 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 615/1265] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:17 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:17.536070 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1ea8f09d-4056-411a-8f3c-c4d97525ef97 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:17.538933 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1ea8f09d-4056-411a-8f3c-c4d97525ef97 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:17.546987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ea8f09d-4056-411a-8f3c-c4d97525ef97 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:17.547628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ea8f09d-4056-411a-8f3c-c4d97525ef97 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:17.548272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ea8f09d-4056-411a-8f3c-c4d97525ef97 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:17.549202 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ea8f09d-4056-411a-8f3c-c4d97525ef97 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:17.549787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ea8f09d-4056-411a-8f3c-c4d97525ef97 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:17.550350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ea8f09d-4056-411a-8f3c-c4d97525ef97 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:17.591809 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1ea8f09d-4056-411a-8f3c-c4d97525ef97 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060241 Aug 07 01:22:17.592523 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 651/1266] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:17 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:18.250799 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1dc45ea8-89ae-431a-ba15-1a3f6914829d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:18.251516 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1dc45ea8-89ae-431a-ba15-1a3f6914829d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:18.259228 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1dc45ea8-89ae-431a-ba15-1a3f6914829d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:18.259811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1dc45ea8-89ae-431a-ba15-1a3f6914829d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:18.260370 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1dc45ea8-89ae-431a-ba15-1a3f6914829d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:18.261299 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1dc45ea8-89ae-431a-ba15-1a3f6914829d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:18.261816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1dc45ea8-89ae-431a-ba15-1a3f6914829d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:18.262373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1dc45ea8-89ae-431a-ba15-1a3f6914829d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:18.359408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1dc45ea8-89ae-431a-ba15-1a3f6914829d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:18.375201 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1dc45ea8-89ae-431a-ba15-1a3f6914829d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.128017 Aug 07 01:22:18.376822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 616/1267] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:18 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:18.609086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d4a9cfc4-170d-4e48-9348-84c65ebaf03c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:18.613045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d4a9cfc4-170d-4e48-9348-84c65ebaf03c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:18.625066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4a9cfc4-170d-4e48-9348-84c65ebaf03c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:18.625952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4a9cfc4-170d-4e48-9348-84c65ebaf03c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:18.626719 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4a9cfc4-170d-4e48-9348-84c65ebaf03c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:18.627986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4a9cfc4-170d-4e48-9348-84c65ebaf03c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:18.628867 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4a9cfc4-170d-4e48-9348-84c65ebaf03c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:18.629612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4a9cfc4-170d-4e48-9348-84c65ebaf03c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:18.689214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d4a9cfc4-170d-4e48-9348-84c65ebaf03c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.085744 Aug 07 01:22:18.689967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 652/1268] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:18 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:19.402243 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ce6e6061-d8b1-460c-a391-7a95b95cd63c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:19.403381 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ce6e6061-d8b1-460c-a391-7a95b95cd63c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:19.415828 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce6e6061-d8b1-460c-a391-7a95b95cd63c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:19.416776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce6e6061-d8b1-460c-a391-7a95b95cd63c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:19.417625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce6e6061-d8b1-460c-a391-7a95b95cd63c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:19.418992 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce6e6061-d8b1-460c-a391-7a95b95cd63c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:19.419862 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce6e6061-d8b1-460c-a391-7a95b95cd63c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:19.420719 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce6e6061-d8b1-460c-a391-7a95b95cd63c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:19.525387 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ce6e6061-d8b1-460c-a391-7a95b95cd63c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:19.534648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ce6e6061-d8b1-460c-a391-7a95b95cd63c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.139463 Aug 07 01:22:19.535227 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 617/1269] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:19 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:19.707774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-194e4f36-cea6-43b1-b057-8ed05b7b1495 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:19.710559 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-194e4f36-cea6-43b1-b057-8ed05b7b1495 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:19.723130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-194e4f36-cea6-43b1-b057-8ed05b7b1495 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:19.724070 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-194e4f36-cea6-43b1-b057-8ed05b7b1495 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:19.724927 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-194e4f36-cea6-43b1-b057-8ed05b7b1495 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:19.726259 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-194e4f36-cea6-43b1-b057-8ed05b7b1495 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:19.727092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-194e4f36-cea6-43b1-b057-8ed05b7b1495 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:19.727872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-194e4f36-cea6-43b1-b057-8ed05b7b1495 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:19.768739 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-194e4f36-cea6-43b1-b057-8ed05b7b1495 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068910 Aug 07 01:22:19.769463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 653/1270] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:19 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:20.572763 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8eb3967c-1cc9-4495-a360-30375ec02c06 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:20.573618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8eb3967c-1cc9-4495-a360-30375ec02c06 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:20.582065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8eb3967c-1cc9-4495-a360-30375ec02c06 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:20.582568 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8eb3967c-1cc9-4495-a360-30375ec02c06 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:20.583016 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8eb3967c-1cc9-4495-a360-30375ec02c06 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:20.583744 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8eb3967c-1cc9-4495-a360-30375ec02c06 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:20.584209 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8eb3967c-1cc9-4495-a360-30375ec02c06 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:20.584677 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8eb3967c-1cc9-4495-a360-30375ec02c06 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:20.667247 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8eb3967c-1cc9-4495-a360-30375ec02c06 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:20.679430 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8eb3967c-1cc9-4495-a360-30375ec02c06 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.112024 Aug 07 01:22:20.680111 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 618/1271] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:20 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:20.786421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-40ad95f0-3aa8-49c9-b660-fe6997fa0cdb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:20.790370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-40ad95f0-3aa8-49c9-b660-fe6997fa0cdb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:20.802800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40ad95f0-3aa8-49c9-b660-fe6997fa0cdb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:20.803693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40ad95f0-3aa8-49c9-b660-fe6997fa0cdb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:20.804553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40ad95f0-3aa8-49c9-b660-fe6997fa0cdb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:20.805894 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40ad95f0-3aa8-49c9-b660-fe6997fa0cdb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:20.806770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40ad95f0-3aa8-49c9-b660-fe6997fa0cdb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:20.807548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40ad95f0-3aa8-49c9-b660-fe6997fa0cdb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:20.867427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-40ad95f0-3aa8-49c9-b660-fe6997fa0cdb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.088811 Aug 07 01:22:20.867960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 654/1272] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:20 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:21.707113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a92de474-c6c4-4e45-8a87-36704d67c0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:21.708221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a92de474-c6c4-4e45-8a87-36704d67c0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:21.724380 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a92de474-c6c4-4e45-8a87-36704d67c0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:21.725281 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a92de474-c6c4-4e45-8a87-36704d67c0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:21.726051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a92de474-c6c4-4e45-8a87-36704d67c0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:21.727343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a92de474-c6c4-4e45-8a87-36704d67c0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:21.728169 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a92de474-c6c4-4e45-8a87-36704d67c0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:21.728979 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a92de474-c6c4-4e45-8a87-36704d67c0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:21.820346 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a92de474-c6c4-4e45-8a87-36704d67c0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:21.830974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a92de474-c6c4-4e45-8a87-36704d67c0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.131885 Aug 07 01:22:21.831662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 619/1273] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:21 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:21.884993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a4ae82c9-312c-410c-bf77-04dc6dff82ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:21.888890 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a4ae82c9-312c-410c-bf77-04dc6dff82ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:21.901168 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4ae82c9-312c-410c-bf77-04dc6dff82ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:21.902036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4ae82c9-312c-410c-bf77-04dc6dff82ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:21.902834 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4ae82c9-312c-410c-bf77-04dc6dff82ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:21.904173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4ae82c9-312c-410c-bf77-04dc6dff82ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:21.905077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4ae82c9-312c-410c-bf77-04dc6dff82ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:21.905874 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4ae82c9-312c-410c-bf77-04dc6dff82ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:21.966373 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a4ae82c9-312c-410c-bf77-04dc6dff82ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.088298 Aug 07 01:22:21.967381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 655/1274] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:21 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:22.854423 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-95941d2d-187f-4a1c-9498-c55c40222dbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:22.855314 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-95941d2d-187f-4a1c-9498-c55c40222dbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:22.863348 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-95941d2d-187f-4a1c-9498-c55c40222dbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:22.863853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-95941d2d-187f-4a1c-9498-c55c40222dbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:22.864347 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-95941d2d-187f-4a1c-9498-c55c40222dbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:22.865105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-95941d2d-187f-4a1c-9498-c55c40222dbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:22.865728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-95941d2d-187f-4a1c-9498-c55c40222dbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:22.866189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-95941d2d-187f-4a1c-9498-c55c40222dbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:22.963587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-95941d2d-187f-4a1c-9498-c55c40222dbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:22.976776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-95941d2d-187f-4a1c-9498-c55c40222dbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.123247 Aug 07 01:22:22.977656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 620/1275] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:22 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:22:22.986547 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a21e66a6-07d9-4ad1-8d3e-173980887f57 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:22.992337 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a21e66a6-07d9-4ad1-8d3e-173980887f57 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:23.013134 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a21e66a6-07d9-4ad1-8d3e-173980887f57 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:23.013983 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a21e66a6-07d9-4ad1-8d3e-173980887f57 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:23.014804 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a21e66a6-07d9-4ad1-8d3e-173980887f57 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:23.016153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a21e66a6-07d9-4ad1-8d3e-173980887f57 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:23.017624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a21e66a6-07d9-4ad1-8d3e-173980887f57 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:23.018424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a21e66a6-07d9-4ad1-8d3e-173980887f57 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:23.057817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a21e66a6-07d9-4ad1-8d3e-173980887f57 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.079075 Aug 07 01:22:23.058475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 656/1276] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:22 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:24.002120 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f942b859-2651-4db4-9b10-678277c2e585 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:24.002755 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f942b859-2651-4db4-9b10-678277c2e585 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:24.009784 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f942b859-2651-4db4-9b10-678277c2e585 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:24.010272 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f942b859-2651-4db4-9b10-678277c2e585 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:24.010707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f942b859-2651-4db4-9b10-678277c2e585 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:24.011419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f942b859-2651-4db4-9b10-678277c2e585 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:24.011863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f942b859-2651-4db4-9b10-678277c2e585 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:24.012316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f942b859-2651-4db4-9b10-678277c2e585 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:24.076045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cee6e979-457f-473a-95ee-9c0ef03b66cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:24.080563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cee6e979-457f-473a-95ee-9c0ef03b66cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:24.083622 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f942b859-2651-4db4-9b10-678277c2e585 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:24.092321 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cee6e979-457f-473a-95ee-9c0ef03b66cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:24.092593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f942b859-2651-4db4-9b10-678277c2e585 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.094269 Aug 07 01:22:24.093122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 621/1277] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:23 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:22:24.093312 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cee6e979-457f-473a-95ee-9c0ef03b66cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:24.093953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cee6e979-457f-473a-95ee-9c0ef03b66cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:24.095885 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cee6e979-457f-473a-95ee-9c0ef03b66cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:24.096767 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cee6e979-457f-473a-95ee-9c0ef03b66cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:24.097537 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cee6e979-457f-473a-95ee-9c0ef03b66cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:24.154064 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cee6e979-457f-473a-95ee-9c0ef03b66cb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.085456 Aug 07 01:22:24.155014 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 657/1278] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:24 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:25.116096 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-94667c9b-1cb0-4098-8af3-3515f03d1df7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:25.117022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-94667c9b-1cb0-4098-8af3-3515f03d1df7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:25.124471 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-94667c9b-1cb0-4098-8af3-3515f03d1df7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:25.125043 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-94667c9b-1cb0-4098-8af3-3515f03d1df7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:25.125523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-94667c9b-1cb0-4098-8af3-3515f03d1df7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:25.127106 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-94667c9b-1cb0-4098-8af3-3515f03d1df7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:25.127629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-94667c9b-1cb0-4098-8af3-3515f03d1df7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:25.128201 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-94667c9b-1cb0-4098-8af3-3515f03d1df7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:25.150905 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:22:25.171073 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-994ab060-f4d7-484c-9ea3-7211b8b52542 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:25.175415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-994ab060-f4d7-484c-9ea3-7211b8b52542 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:25.185492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-994ab060-f4d7-484c-9ea3-7211b8b52542 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:25.186081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-994ab060-f4d7-484c-9ea3-7211b8b52542 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:25.186600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-994ab060-f4d7-484c-9ea3-7211b8b52542 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:25.187349 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-994ab060-f4d7-484c-9ea3-7211b8b52542 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:25.187815 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-994ab060-f4d7-484c-9ea3-7211b8b52542 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:25.188270 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-994ab060-f4d7-484c-9ea3-7211b8b52542 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:25.212230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-94667c9b-1cb0-4098-8af3-3515f03d1df7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:25.224732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-994ab060-f4d7-484c-9ea3-7211b8b52542 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060570 Aug 07 01:22:25.225407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 658/1279] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:25 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:25.227558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-94667c9b-1cb0-4098-8af3-3515f03d1df7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.115211 Aug 07 01:22:25.228679 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 622/1280] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:25 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:26.239242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e7f5f4e4-953e-4c45-8ce9-5b14da605f13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:26.241777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e7f5f4e4-953e-4c45-8ce9-5b14da605f13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:26.250428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e7f5f4e4-953e-4c45-8ce9-5b14da605f13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:26.250991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e7f5f4e4-953e-4c45-8ce9-5b14da605f13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:26.251921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e7f5f4e4-953e-4c45-8ce9-5b14da605f13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:26.252707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e7f5f4e4-953e-4c45-8ce9-5b14da605f13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:26.253169 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e7f5f4e4-953e-4c45-8ce9-5b14da605f13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:26.253622 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e7f5f4e4-953e-4c45-8ce9-5b14da605f13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:26.256537 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f0cbccb9-84ca-4bdf-a866-ccc0a856d07f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:26.257698 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f0cbccb9-84ca-4bdf-a866-ccc0a856d07f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:26.269935 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0cbccb9-84ca-4bdf-a866-ccc0a856d07f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:26.270783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0cbccb9-84ca-4bdf-a866-ccc0a856d07f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:26.271551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0cbccb9-84ca-4bdf-a866-ccc0a856d07f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:26.272872 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0cbccb9-84ca-4bdf-a866-ccc0a856d07f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:26.273662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0cbccb9-84ca-4bdf-a866-ccc0a856d07f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:26.274405 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0cbccb9-84ca-4bdf-a866-ccc0a856d07f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:26.304545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e7f5f4e4-953e-4c45-8ce9-5b14da605f13 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.070211 Aug 07 01:22:26.311176 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 659/1281] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:26 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:26.377724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f0cbccb9-84ca-4bdf-a866-ccc0a856d07f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:26.401204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f0cbccb9-84ca-4bdf-a866-ccc0a856d07f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.152160 Aug 07 01:22:26.402326 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 623/1282] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:26 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:26.930385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:22:26.930721 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:22:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:22:27.326334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-12007ccb-d52b-40a4-9fcd-6469e2112142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:27.330447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-12007ccb-d52b-40a4-9fcd-6469e2112142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:27.345879 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-12007ccb-d52b-40a4-9fcd-6469e2112142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:27.346748 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-12007ccb-d52b-40a4-9fcd-6469e2112142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:27.347439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-12007ccb-d52b-40a4-9fcd-6469e2112142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:27.348626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-12007ccb-d52b-40a4-9fcd-6469e2112142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:27.349147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-12007ccb-d52b-40a4-9fcd-6469e2112142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:27.349702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-12007ccb-d52b-40a4-9fcd-6469e2112142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:27.400703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-12007ccb-d52b-40a4-9fcd-6469e2112142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.083187 Aug 07 01:22:27.401566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 660/1283] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:27 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:27.428463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-efdb4d95-9b9b-4745-9c85-9a5b2d570ac1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:27.429813 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-efdb4d95-9b9b-4745-9c85-9a5b2d570ac1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:27.443818 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-efdb4d95-9b9b-4745-9c85-9a5b2d570ac1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:27.444278 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-efdb4d95-9b9b-4745-9c85-9a5b2d570ac1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:27.445093 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-efdb4d95-9b9b-4745-9c85-9a5b2d570ac1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:27.446426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-efdb4d95-9b9b-4745-9c85-9a5b2d570ac1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:27.447281 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-efdb4d95-9b9b-4745-9c85-9a5b2d570ac1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:27.448077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-efdb4d95-9b9b-4745-9c85-9a5b2d570ac1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:27.549826 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-efdb4d95-9b9b-4745-9c85-9a5b2d570ac1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:27.559005 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-efdb4d95-9b9b-4745-9c85-9a5b2d570ac1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.138402 Aug 07 01:22:27.559647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 624/1284] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:27 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:27.996634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:22:27.996866 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:22:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:22:28.416510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-94c596c5-65bd-4cea-8d09-e64ee0e79ce7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:28.418792 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-94c596c5-65bd-4cea-8d09-e64ee0e79ce7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:28.426181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94c596c5-65bd-4cea-8d09-e64ee0e79ce7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:28.426668 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94c596c5-65bd-4cea-8d09-e64ee0e79ce7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:28.427103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94c596c5-65bd-4cea-8d09-e64ee0e79ce7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:28.427866 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94c596c5-65bd-4cea-8d09-e64ee0e79ce7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:28.428365 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94c596c5-65bd-4cea-8d09-e64ee0e79ce7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:28.428803 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94c596c5-65bd-4cea-8d09-e64ee0e79ce7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:28.465135 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-94c596c5-65bd-4cea-8d09-e64ee0e79ce7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052787 Aug 07 01:22:28.465753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 661/1285] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:28 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:28.582592 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-89529c9a-a1ec-4114-a9f8-7e5215a9a43f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:28.583195 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-89529c9a-a1ec-4114-a9f8-7e5215a9a43f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:28.590428 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89529c9a-a1ec-4114-a9f8-7e5215a9a43f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:28.590923 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89529c9a-a1ec-4114-a9f8-7e5215a9a43f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:28.591363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89529c9a-a1ec-4114-a9f8-7e5215a9a43f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:28.592126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89529c9a-a1ec-4114-a9f8-7e5215a9a43f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:28.592625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89529c9a-a1ec-4114-a9f8-7e5215a9a43f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:28.593051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89529c9a-a1ec-4114-a9f8-7e5215a9a43f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:28.676966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-89529c9a-a1ec-4114-a9f8-7e5215a9a43f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:28.685216 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-89529c9a-a1ec-4114-a9f8-7e5215a9a43f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.107470 Aug 07 01:22:28.685875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 625/1286] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:28 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:29.479578 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3a2c26cf-0c44-464b-8b2d-9f07307b8faa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:29.483465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3a2c26cf-0c44-464b-8b2d-9f07307b8faa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:29.492559 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a2c26cf-0c44-464b-8b2d-9f07307b8faa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:29.493092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a2c26cf-0c44-464b-8b2d-9f07307b8faa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:29.493578 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a2c26cf-0c44-464b-8b2d-9f07307b8faa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:29.494323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a2c26cf-0c44-464b-8b2d-9f07307b8faa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:29.494819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a2c26cf-0c44-464b-8b2d-9f07307b8faa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:29.495244 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3a2c26cf-0c44-464b-8b2d-9f07307b8faa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:29.532793 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3a2c26cf-0c44-464b-8b2d-9f07307b8faa tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058702 Aug 07 01:22:29.533368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 662/1287] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:29 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:29.707135 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1ce8e6c7-8bf2-44ac-a696-d7d514e3d53f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:29.707790 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1ce8e6c7-8bf2-44ac-a696-d7d514e3d53f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:29.714534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ce8e6c7-8bf2-44ac-a696-d7d514e3d53f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:29.715070 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ce8e6c7-8bf2-44ac-a696-d7d514e3d53f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:29.715534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ce8e6c7-8bf2-44ac-a696-d7d514e3d53f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:29.716278 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ce8e6c7-8bf2-44ac-a696-d7d514e3d53f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:29.716729 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ce8e6c7-8bf2-44ac-a696-d7d514e3d53f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:29.717155 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1ce8e6c7-8bf2-44ac-a696-d7d514e3d53f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:29.787429 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1ce8e6c7-8bf2-44ac-a696-d7d514e3d53f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:29.795769 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1ce8e6c7-8bf2-44ac-a696-d7d514e3d53f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.093134 Aug 07 01:22:29.796707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 626/1288] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:29 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:30.547984 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5d83e1c4-3feb-4a90-81b8-7e76529c97fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:30.550147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5d83e1c4-3feb-4a90-81b8-7e76529c97fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:30.557655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d83e1c4-3feb-4a90-81b8-7e76529c97fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:30.558149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d83e1c4-3feb-4a90-81b8-7e76529c97fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:30.558592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d83e1c4-3feb-4a90-81b8-7e76529c97fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:30.559347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d83e1c4-3feb-4a90-81b8-7e76529c97fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:30.559811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d83e1c4-3feb-4a90-81b8-7e76529c97fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:30.560276 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d83e1c4-3feb-4a90-81b8-7e76529c97fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:30.592315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:22:30.598780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5d83e1c4-3feb-4a90-81b8-7e76529c97fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055384 Aug 07 01:22:30.599389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 663/1289] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:30 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:30.818335 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c5677207-0946-47cf-84bb-c0594fee04b2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:30.819443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c5677207-0946-47cf-84bb-c0594fee04b2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:30.826349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c5677207-0946-47cf-84bb-c0594fee04b2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:30.826847 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c5677207-0946-47cf-84bb-c0594fee04b2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:30.827282 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c5677207-0946-47cf-84bb-c0594fee04b2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:30.828006 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c5677207-0946-47cf-84bb-c0594fee04b2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:30.828501 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c5677207-0946-47cf-84bb-c0594fee04b2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:30.829471 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c5677207-0946-47cf-84bb-c0594fee04b2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:30.900940 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c5677207-0946-47cf-84bb-c0594fee04b2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:30.914634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c5677207-0946-47cf-84bb-c0594fee04b2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100454 Aug 07 01:22:30.915491 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 627/1290] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:30 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:31.613977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f25b0071-edda-4fa3-a3f9-05ae14af98fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:31.616585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f25b0071-edda-4fa3-a3f9-05ae14af98fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:31.625595 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f25b0071-edda-4fa3-a3f9-05ae14af98fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:31.627014 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f25b0071-edda-4fa3-a3f9-05ae14af98fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:31.627820 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f25b0071-edda-4fa3-a3f9-05ae14af98fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:31.629258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f25b0071-edda-4fa3-a3f9-05ae14af98fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:31.630098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f25b0071-edda-4fa3-a3f9-05ae14af98fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:31.630891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f25b0071-edda-4fa3-a3f9-05ae14af98fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:31.676496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f25b0071-edda-4fa3-a3f9-05ae14af98fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068160 Aug 07 01:22:31.677459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 664/1291] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:31 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:31.954996 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3fb26405-1d60-477b-aaf6-1748a2be59bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:31.956066 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3fb26405-1d60-477b-aaf6-1748a2be59bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:31.969447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fb26405-1d60-477b-aaf6-1748a2be59bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:31.970331 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fb26405-1d60-477b-aaf6-1748a2be59bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:31.971125 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fb26405-1d60-477b-aaf6-1748a2be59bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:31.972479 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fb26405-1d60-477b-aaf6-1748a2be59bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:31.973348 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fb26405-1d60-477b-aaf6-1748a2be59bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:31.974196 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fb26405-1d60-477b-aaf6-1748a2be59bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:32.089945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3fb26405-1d60-477b-aaf6-1748a2be59bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:32.103710 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3fb26405-1d60-477b-aaf6-1748a2be59bc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.157130 Aug 07 01:22:32.104679 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 628/1292] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:31 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:32.693943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-aab09e1c-8f70-455f-b5bb-69786cf27d1e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:32.697884 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-aab09e1c-8f70-455f-b5bb-69786cf27d1e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:32.711246 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aab09e1c-8f70-455f-b5bb-69786cf27d1e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:32.712078 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aab09e1c-8f70-455f-b5bb-69786cf27d1e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:32.712884 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aab09e1c-8f70-455f-b5bb-69786cf27d1e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:32.714123 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aab09e1c-8f70-455f-b5bb-69786cf27d1e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:32.714934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aab09e1c-8f70-455f-b5bb-69786cf27d1e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:32.715700 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aab09e1c-8f70-455f-b5bb-69786cf27d1e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:32.777504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-aab09e1c-8f70-455f-b5bb-69786cf27d1e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.090047 Aug 07 01:22:32.778563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 665/1293] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:32 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:33.142536 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bee95156-beed-4617-a1a0-01056b67af8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:33.143668 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bee95156-beed-4617-a1a0-01056b67af8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:33.155902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bee95156-beed-4617-a1a0-01056b67af8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:33.156795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bee95156-beed-4617-a1a0-01056b67af8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:33.157638 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bee95156-beed-4617-a1a0-01056b67af8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:33.158990 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bee95156-beed-4617-a1a0-01056b67af8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:33.159798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bee95156-beed-4617-a1a0-01056b67af8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:33.160613 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bee95156-beed-4617-a1a0-01056b67af8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:33.261130 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bee95156-beed-4617-a1a0-01056b67af8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:33.270766 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bee95156-beed-4617-a1a0-01056b67af8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.136123 Aug 07 01:22:33.271476 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 629/1294] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:33 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:33.796694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-36350774-1fc9-4538-a657-25296aecde2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:33.801924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-36350774-1fc9-4538-a657-25296aecde2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:33.805971 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=56 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:22:33.816958 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36350774-1fc9-4538-a657-25296aecde2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:33.817874 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36350774-1fc9-4538-a657-25296aecde2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:33.818653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36350774-1fc9-4538-a657-25296aecde2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:33.819611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36350774-1fc9-4538-a657-25296aecde2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:33.820498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36350774-1fc9-4538-a657-25296aecde2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:33.821285 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36350774-1fc9-4538-a657-25296aecde2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:33.883832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-36350774-1fc9-4538-a657-25296aecde2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.095145 Aug 07 01:22:33.884849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 666/1295] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:33 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:34.302516 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c651a5e8-3aaf-48f5-aa47-396a939aaba6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:34.303704 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c651a5e8-3aaf-48f5-aa47-396a939aaba6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:34.316725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c651a5e8-3aaf-48f5-aa47-396a939aaba6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:34.317587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c651a5e8-3aaf-48f5-aa47-396a939aaba6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:34.319426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c651a5e8-3aaf-48f5-aa47-396a939aaba6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:34.320862 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c651a5e8-3aaf-48f5-aa47-396a939aaba6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:34.321869 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c651a5e8-3aaf-48f5-aa47-396a939aaba6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:34.322644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c651a5e8-3aaf-48f5-aa47-396a939aaba6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:34.442168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c651a5e8-3aaf-48f5-aa47-396a939aaba6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:34.455848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c651a5e8-3aaf-48f5-aa47-396a939aaba6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.160361 Aug 07 01:22:34.455848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 630/1296] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:34 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:34.901197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f40aa774-c804-494f-b85e-adb74c8ebe95 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:34.903492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f40aa774-c804-494f-b85e-adb74c8ebe95 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:34.910682 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f40aa774-c804-494f-b85e-adb74c8ebe95 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:34.911188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f40aa774-c804-494f-b85e-adb74c8ebe95 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:34.911632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f40aa774-c804-494f-b85e-adb74c8ebe95 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:34.912399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f40aa774-c804-494f-b85e-adb74c8ebe95 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:34.912872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f40aa774-c804-494f-b85e-adb74c8ebe95 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:34.913302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f40aa774-c804-494f-b85e-adb74c8ebe95 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:34.955996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f40aa774-c804-494f-b85e-adb74c8ebe95 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059232 Aug 07 01:22:34.956628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 667/1297] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:34 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:35.477921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7a5b9ed1-6701-4e7f-942a-64eaa2634e9d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:35.478796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7a5b9ed1-6701-4e7f-942a-64eaa2634e9d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:35.481171 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=52 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:22:35.487302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a5b9ed1-6701-4e7f-942a-64eaa2634e9d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:35.487796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a5b9ed1-6701-4e7f-942a-64eaa2634e9d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:35.488267 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a5b9ed1-6701-4e7f-942a-64eaa2634e9d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:35.489013 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a5b9ed1-6701-4e7f-942a-64eaa2634e9d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:35.489473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a5b9ed1-6701-4e7f-942a-64eaa2634e9d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:35.489910 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7a5b9ed1-6701-4e7f-942a-64eaa2634e9d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:35.586626 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7a5b9ed1-6701-4e7f-942a-64eaa2634e9d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:35.596669 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7a5b9ed1-6701-4e7f-942a-64eaa2634e9d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.123241 Aug 07 01:22:35.597447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 631/1298] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:35 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:22:35.969022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-72fea05a-b967-44f6-891b-34d744c8548b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:35.972146 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-72fea05a-b967-44f6-891b-34d744c8548b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:35.979479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72fea05a-b967-44f6-891b-34d744c8548b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:35.979974 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72fea05a-b967-44f6-891b-34d744c8548b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:35.980466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72fea05a-b967-44f6-891b-34d744c8548b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:35.981209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72fea05a-b967-44f6-891b-34d744c8548b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:35.981674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72fea05a-b967-44f6-891b-34d744c8548b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:35.982095 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72fea05a-b967-44f6-891b-34d744c8548b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:36.018506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-72fea05a-b967-44f6-891b-34d744c8548b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053539 Aug 07 01:22:36.019265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 668/1299] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:35 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:36.629007 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4d7b8122-492c-44e2-87ed-5bf05fafd0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:36.629622 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4d7b8122-492c-44e2-87ed-5bf05fafd0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:36.636687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d7b8122-492c-44e2-87ed-5bf05fafd0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:36.637186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d7b8122-492c-44e2-87ed-5bf05fafd0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:36.637629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d7b8122-492c-44e2-87ed-5bf05fafd0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:36.638360 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d7b8122-492c-44e2-87ed-5bf05fafd0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:36.638821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d7b8122-492c-44e2-87ed-5bf05fafd0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:36.639253 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d7b8122-492c-44e2-87ed-5bf05fafd0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:36.725776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4d7b8122-492c-44e2-87ed-5bf05fafd0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:36.733716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4d7b8122-492c-44e2-87ed-5bf05fafd0a2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.109381 Aug 07 01:22:36.734310 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 632/1300] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:36 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:37.037870 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e66fefc9-7d9a-4af5-a3d5-1237bf97f36f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:37.041845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e66fefc9-7d9a-4af5-a3d5-1237bf97f36f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:37.055180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e66fefc9-7d9a-4af5-a3d5-1237bf97f36f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:37.056107 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e66fefc9-7d9a-4af5-a3d5-1237bf97f36f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:37.056934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e66fefc9-7d9a-4af5-a3d5-1237bf97f36f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:37.058243 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e66fefc9-7d9a-4af5-a3d5-1237bf97f36f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:37.059090 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e66fefc9-7d9a-4af5-a3d5-1237bf97f36f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:37.059828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e66fefc9-7d9a-4af5-a3d5-1237bf97f36f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:37.120225 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e66fefc9-7d9a-4af5-a3d5-1237bf97f36f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.090558 Aug 07 01:22:37.121655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 669/1301] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:37 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:37.760128 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ffbc89c2-9067-4124-b8c7-31a3b590f27b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:37.761292 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ffbc89c2-9067-4124-b8c7-31a3b590f27b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:37.773463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffbc89c2-9067-4124-b8c7-31a3b590f27b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:37.774329 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffbc89c2-9067-4124-b8c7-31a3b590f27b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:37.775182 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffbc89c2-9067-4124-b8c7-31a3b590f27b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:37.776544 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffbc89c2-9067-4124-b8c7-31a3b590f27b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:37.777286 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffbc89c2-9067-4124-b8c7-31a3b590f27b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:37.777796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffbc89c2-9067-4124-b8c7-31a3b590f27b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:37.892758 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ffbc89c2-9067-4124-b8c7-31a3b590f27b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:37.908316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ffbc89c2-9067-4124-b8c7-31a3b590f27b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.154635 Aug 07 01:22:37.909391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 633/1302] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:37 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:38.141845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-94f49f22-6a4e-46f7-a51f-047834143d16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:38.145736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-94f49f22-6a4e-46f7-a51f-047834143d16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:38.155990 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94f49f22-6a4e-46f7-a51f-047834143d16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:38.156900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94f49f22-6a4e-46f7-a51f-047834143d16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:38.157515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94f49f22-6a4e-46f7-a51f-047834143d16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:38.158335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94f49f22-6a4e-46f7-a51f-047834143d16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:38.158813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94f49f22-6a4e-46f7-a51f-047834143d16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:38.159258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94f49f22-6a4e-46f7-a51f-047834143d16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:38.201191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-94f49f22-6a4e-46f7-a51f-047834143d16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067682 Aug 07 01:22:38.201943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 670/1303] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:38 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:38.947014 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fe9f8ca1-b61e-4c56-b5ef-8b76d0ae988e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:38.948005 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fe9f8ca1-b61e-4c56-b5ef-8b76d0ae988e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:38.958087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe9f8ca1-b61e-4c56-b5ef-8b76d0ae988e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:38.958587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe9f8ca1-b61e-4c56-b5ef-8b76d0ae988e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:38.959108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe9f8ca1-b61e-4c56-b5ef-8b76d0ae988e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:38.959797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe9f8ca1-b61e-4c56-b5ef-8b76d0ae988e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:38.960303 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe9f8ca1-b61e-4c56-b5ef-8b76d0ae988e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:38.960744 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe9f8ca1-b61e-4c56-b5ef-8b76d0ae988e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:39.049132 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fe9f8ca1-b61e-4c56-b5ef-8b76d0ae988e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:39.059045 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fe9f8ca1-b61e-4c56-b5ef-8b76d0ae988e tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.116723 Aug 07 01:22:39.059801 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 634/1304] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:38 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:39.223277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-617768d6-1c3b-4a82-9ac9-4506232729a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:39.227483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-617768d6-1c3b-4a82-9ac9-4506232729a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:39.241877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-617768d6-1c3b-4a82-9ac9-4506232729a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:39.242732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-617768d6-1c3b-4a82-9ac9-4506232729a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:39.243711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-617768d6-1c3b-4a82-9ac9-4506232729a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:39.245516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-617768d6-1c3b-4a82-9ac9-4506232729a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:39.246648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-617768d6-1c3b-4a82-9ac9-4506232729a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:39.247572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-617768d6-1c3b-4a82-9ac9-4506232729a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:39.314908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-617768d6-1c3b-4a82-9ac9-4506232729a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.100213 Aug 07 01:22:39.315758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 671/1305] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:39 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:40.100062 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d51a72d6-6c88-458a-af4b-1fe66575a87a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:40.101229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d51a72d6-6c88-458a-af4b-1fe66575a87a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:40.109845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d51a72d6-6c88-458a-af4b-1fe66575a87a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:40.110490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d51a72d6-6c88-458a-af4b-1fe66575a87a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:40.111082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d51a72d6-6c88-458a-af4b-1fe66575a87a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:40.111917 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d51a72d6-6c88-458a-af4b-1fe66575a87a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:40.112487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d51a72d6-6c88-458a-af4b-1fe66575a87a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:40.113179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d51a72d6-6c88-458a-af4b-1fe66575a87a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:40.206284 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d51a72d6-6c88-458a-af4b-1fe66575a87a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:40.216727 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d51a72d6-6c88-458a-af4b-1fe66575a87a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.123379 Aug 07 01:22:40.217797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 635/1306] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:40 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:40.332891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-38bef354-3268-48b0-a689-435086270aed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:40.335678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-38bef354-3268-48b0-a689-435086270aed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:40.342972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38bef354-3268-48b0-a689-435086270aed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:40.343470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38bef354-3268-48b0-a689-435086270aed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:40.344754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38bef354-3268-48b0-a689-435086270aed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:40.345655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38bef354-3268-48b0-a689-435086270aed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:40.346171 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38bef354-3268-48b0-a689-435086270aed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:40.346607 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38bef354-3268-48b0-a689-435086270aed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:40.382978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-38bef354-3268-48b0-a689-435086270aed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056888 Aug 07 01:22:40.383732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 672/1307] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:40 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:41.254609 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-65b5cea4-67dc-42cf-931f-7826de40f2d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:41.255655 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-65b5cea4-67dc-42cf-931f-7826de40f2d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:41.267739 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65b5cea4-67dc-42cf-931f-7826de40f2d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:41.268659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65b5cea4-67dc-42cf-931f-7826de40f2d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:41.269470 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65b5cea4-67dc-42cf-931f-7826de40f2d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:41.270799 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65b5cea4-67dc-42cf-931f-7826de40f2d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:41.271643 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65b5cea4-67dc-42cf-931f-7826de40f2d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:41.272487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65b5cea4-67dc-42cf-931f-7826de40f2d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:41.397285 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bc8794c5-956d-4972-adac-fea280e5b14e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:41.398671 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-65b5cea4-67dc-42cf-931f-7826de40f2d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:41.400553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bc8794c5-956d-4972-adac-fea280e5b14e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:41.409489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc8794c5-956d-4972-adac-fea280e5b14e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:41.410342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc8794c5-956d-4972-adac-fea280e5b14e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:41.411087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc8794c5-956d-4972-adac-fea280e5b14e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:41.411882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc8794c5-956d-4972-adac-fea280e5b14e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:41.412425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc8794c5-956d-4972-adac-fea280e5b14e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:41.412612 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-65b5cea4-67dc-42cf-931f-7826de40f2d2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.162408 Aug 07 01:22:41.412971 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc8794c5-956d-4972-adac-fea280e5b14e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:41.413612 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 636/1308] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:41 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:41.465059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bc8794c5-956d-4972-adac-fea280e5b14e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073010 Aug 07 01:22:41.466138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 673/1309] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:41 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:42.459427 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f52d777f-9af7-4d23-bd84-e02839b2cfeb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:42.461200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f52d777f-9af7-4d23-bd84-e02839b2cfeb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:42.469236 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f52d777f-9af7-4d23-bd84-e02839b2cfeb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:42.469800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f52d777f-9af7-4d23-bd84-e02839b2cfeb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:42.470253 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f52d777f-9af7-4d23-bd84-e02839b2cfeb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:42.470992 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f52d777f-9af7-4d23-bd84-e02839b2cfeb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:42.471453 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f52d777f-9af7-4d23-bd84-e02839b2cfeb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:42.471883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f52d777f-9af7-4d23-bd84-e02839b2cfeb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:42.485367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-412de266-0e68-4b0a-b97b-6f5ca94818b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:42.489479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-412de266-0e68-4b0a-b97b-6f5ca94818b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:42.501399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-412de266-0e68-4b0a-b97b-6f5ca94818b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:42.502165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-412de266-0e68-4b0a-b97b-6f5ca94818b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:42.502956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-412de266-0e68-4b0a-b97b-6f5ca94818b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:42.504207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-412de266-0e68-4b0a-b97b-6f5ca94818b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:42.505092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-412de266-0e68-4b0a-b97b-6f5ca94818b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:42.505862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-412de266-0e68-4b0a-b97b-6f5ca94818b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:42.559834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f52d777f-9af7-4d23-bd84-e02839b2cfeb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:42.565967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-412de266-0e68-4b0a-b97b-6f5ca94818b9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.088914 Aug 07 01:22:42.567022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 674/1310] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:42 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:42.569086 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f52d777f-9af7-4d23-bd84-e02839b2cfeb tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.115952 Aug 07 01:22:42.569671 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 637/1311] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:42 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:43.584663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9c8e45c1-0c54-4b82-8889-537312e1438a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:43.587957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9c8e45c1-0c54-4b82-8889-537312e1438a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:43.591105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-09ece523-5003-4b0f-bfac-4ede9cd54abc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:43.591720 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-09ece523-5003-4b0f-bfac-4ede9cd54abc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:43.599042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09ece523-5003-4b0f-bfac-4ede9cd54abc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:43.599302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9c8e45c1-0c54-4b82-8889-537312e1438a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:43.599532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09ece523-5003-4b0f-bfac-4ede9cd54abc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:43.599690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9c8e45c1-0c54-4b82-8889-537312e1438a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:43.600023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9c8e45c1-0c54-4b82-8889-537312e1438a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:43.600375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09ece523-5003-4b0f-bfac-4ede9cd54abc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:43.601248 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9c8e45c1-0c54-4b82-8889-537312e1438a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:43.601373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09ece523-5003-4b0f-bfac-4ede9cd54abc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:43.601750 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09ece523-5003-4b0f-bfac-4ede9cd54abc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:43.601960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9c8e45c1-0c54-4b82-8889-537312e1438a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:43.602192 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09ece523-5003-4b0f-bfac-4ede9cd54abc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:43.602596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9c8e45c1-0c54-4b82-8889-537312e1438a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:43.650781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9c8e45c1-0c54-4b82-8889-537312e1438a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.072694 Aug 07 01:22:43.651568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 675/1312] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:43 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:43.703259 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-09ece523-5003-4b0f-bfac-4ede9cd54abc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:43.711583 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-09ece523-5003-4b0f-bfac-4ede9cd54abc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.124406 Aug 07 01:22:43.712175 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 638/1313] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:43 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:22:44.667729 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2994eedf-45af-4728-b5be-ac83adcc6309 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:44.670157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2994eedf-45af-4728-b5be-ac83adcc6309 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:44.678464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2994eedf-45af-4728-b5be-ac83adcc6309 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:44.679148 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2994eedf-45af-4728-b5be-ac83adcc6309 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:44.679618 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2994eedf-45af-4728-b5be-ac83adcc6309 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:44.680504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2994eedf-45af-4728-b5be-ac83adcc6309 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:44.681044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2994eedf-45af-4728-b5be-ac83adcc6309 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:44.681640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2994eedf-45af-4728-b5be-ac83adcc6309 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:44.740313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2994eedf-45af-4728-b5be-ac83adcc6309 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.078377 Aug 07 01:22:44.741386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 676/1314] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:44 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:44.751935 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5dcafe2a-3e12-4191-9a60-b07ddbf92033 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:44.753121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5dcafe2a-3e12-4191-9a60-b07ddbf92033 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:44.765298 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5dcafe2a-3e12-4191-9a60-b07ddbf92033 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:44.766182 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5dcafe2a-3e12-4191-9a60-b07ddbf92033 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:44.766993 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5dcafe2a-3e12-4191-9a60-b07ddbf92033 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:44.768368 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5dcafe2a-3e12-4191-9a60-b07ddbf92033 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:44.769239 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5dcafe2a-3e12-4191-9a60-b07ddbf92033 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:44.770025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5dcafe2a-3e12-4191-9a60-b07ddbf92033 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:44.778616 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:22:44.864173 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5dcafe2a-3e12-4191-9a60-b07ddbf92033 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:44.872858 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5dcafe2a-3e12-4191-9a60-b07ddbf92033 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.128388 Aug 07 01:22:44.873451 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 639/1315] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:44 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:45.758507 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7cecf83a-0ffd-4ba3-afbd-a59bba48604f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:45.761399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7cecf83a-0ffd-4ba3-afbd-a59bba48604f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:45.770263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7cecf83a-0ffd-4ba3-afbd-a59bba48604f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:45.770768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7cecf83a-0ffd-4ba3-afbd-a59bba48604f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:45.771244 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7cecf83a-0ffd-4ba3-afbd-a59bba48604f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:45.771992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7cecf83a-0ffd-4ba3-afbd-a59bba48604f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:45.772502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7cecf83a-0ffd-4ba3-afbd-a59bba48604f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:45.773082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7cecf83a-0ffd-4ba3-afbd-a59bba48604f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:45.827038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7cecf83a-0ffd-4ba3-afbd-a59bba48604f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.072820 Aug 07 01:22:45.827790 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 677/1316] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:45 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:45.894945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3fa6e00a-87fa-4cb9-9110-eeee8dfa396b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:45.895802 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3fa6e00a-87fa-4cb9-9110-eeee8dfa396b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:45.904831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fa6e00a-87fa-4cb9-9110-eeee8dfa396b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:45.905382 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fa6e00a-87fa-4cb9-9110-eeee8dfa396b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:45.905936 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fa6e00a-87fa-4cb9-9110-eeee8dfa396b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:45.906801 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fa6e00a-87fa-4cb9-9110-eeee8dfa396b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:45.907388 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fa6e00a-87fa-4cb9-9110-eeee8dfa396b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:45.908034 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3fa6e00a-87fa-4cb9-9110-eeee8dfa396b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:46.011653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3fa6e00a-87fa-4cb9-9110-eeee8dfa396b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:46.023781 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3fa6e00a-87fa-4cb9-9110-eeee8dfa396b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.133535 Aug 07 01:22:46.024860 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 640/1317] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:45 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:46.843313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7b4ed08c-6738-433e-a72d-7ab2be4299f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:46.847601 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7b4ed08c-6738-433e-a72d-7ab2be4299f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:46.860066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b4ed08c-6738-433e-a72d-7ab2be4299f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:46.860986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b4ed08c-6738-433e-a72d-7ab2be4299f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:46.861778 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b4ed08c-6738-433e-a72d-7ab2be4299f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:46.863095 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b4ed08c-6738-433e-a72d-7ab2be4299f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:46.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b4ed08c-6738-433e-a72d-7ab2be4299f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:46.864783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7b4ed08c-6738-433e-a72d-7ab2be4299f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:46.905083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7b4ed08c-6738-433e-a72d-7ab2be4299f5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066096 Aug 07 01:22:46.905679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 678/1318] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:46 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:47.060343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5c93f514-a9fe-4d9a-821d-0a053411898d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:47.061246 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5c93f514-a9fe-4d9a-821d-0a053411898d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:47.072273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c93f514-a9fe-4d9a-821d-0a053411898d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:47.072970 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c93f514-a9fe-4d9a-821d-0a053411898d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:47.073615 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c93f514-a9fe-4d9a-821d-0a053411898d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:47.075137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c93f514-a9fe-4d9a-821d-0a053411898d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:47.076193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c93f514-a9fe-4d9a-821d-0a053411898d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:47.077258 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5c93f514-a9fe-4d9a-821d-0a053411898d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:47.150942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5c93f514-a9fe-4d9a-821d-0a053411898d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:47.160065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5c93f514-a9fe-4d9a-821d-0a053411898d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.103994 Aug 07 01:22:47.160797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 641/1319] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:47 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:22:47.922938 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1a872843-27a0-4ed8-b464-b21a830475a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:47.926590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1a872843-27a0-4ed8-b464-b21a830475a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:47.939461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a872843-27a0-4ed8-b464-b21a830475a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:47.940420 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a872843-27a0-4ed8-b464-b21a830475a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:47.941268 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a872843-27a0-4ed8-b464-b21a830475a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:47.942555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a872843-27a0-4ed8-b464-b21a830475a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:47.943428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a872843-27a0-4ed8-b464-b21a830475a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:47.944261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a872843-27a0-4ed8-b464-b21a830475a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:48.006671 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1a872843-27a0-4ed8-b464-b21a830475a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.091518 Aug 07 01:22:48.007849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 679/1320] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:47 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:48.183168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ab5f0f7b-8ad5-4d63-9cb0-7dde5c13d445 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:48.183825 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ab5f0f7b-8ad5-4d63-9cb0-7dde5c13d445 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:48.191481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab5f0f7b-8ad5-4d63-9cb0-7dde5c13d445 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:48.191970 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab5f0f7b-8ad5-4d63-9cb0-7dde5c13d445 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:48.192449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab5f0f7b-8ad5-4d63-9cb0-7dde5c13d445 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:48.193184 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab5f0f7b-8ad5-4d63-9cb0-7dde5c13d445 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:48.193639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab5f0f7b-8ad5-4d63-9cb0-7dde5c13d445 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:48.194076 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab5f0f7b-8ad5-4d63-9cb0-7dde5c13d445 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:48.276942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ab5f0f7b-8ad5-4d63-9cb0-7dde5c13d445 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:48.286315 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ab5f0f7b-8ad5-4d63-9cb0-7dde5c13d445 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.107571 Aug 07 01:22:48.286915 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 642/1321] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:48 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:49.024210 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-36de1cf9-aa5a-4e96-8126-badebb1e4aae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:49.027446 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-36de1cf9-aa5a-4e96-8126-badebb1e4aae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:49.037359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36de1cf9-aa5a-4e96-8126-badebb1e4aae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:49.037973 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36de1cf9-aa5a-4e96-8126-badebb1e4aae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:49.038980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36de1cf9-aa5a-4e96-8126-badebb1e4aae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:49.039406 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36de1cf9-aa5a-4e96-8126-badebb1e4aae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:49.039942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36de1cf9-aa5a-4e96-8126-badebb1e4aae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:49.040465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36de1cf9-aa5a-4e96-8126-badebb1e4aae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:49.061927 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:22:49.077857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-36de1cf9-aa5a-4e96-8126-badebb1e4aae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058214 Aug 07 01:22:49.079545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 680/1322] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:49 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:49.318225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-84eacb30-227e-49c4-8627-4ba482ccd7b9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:49.318980 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-84eacb30-227e-49c4-8627-4ba482ccd7b9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:49.327858 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84eacb30-227e-49c4-8627-4ba482ccd7b9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:49.328637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84eacb30-227e-49c4-8627-4ba482ccd7b9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:49.329083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84eacb30-227e-49c4-8627-4ba482ccd7b9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:49.329884 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84eacb30-227e-49c4-8627-4ba482ccd7b9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:49.330355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84eacb30-227e-49c4-8627-4ba482ccd7b9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:49.330828 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-84eacb30-227e-49c4-8627-4ba482ccd7b9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:49.420816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-84eacb30-227e-49c4-8627-4ba482ccd7b9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:49.435336 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-84eacb30-227e-49c4-8627-4ba482ccd7b9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.120706 Aug 07 01:22:49.436388 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 643/1323] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:49 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:22:50.091807 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6dbe20b0-ae04-4b1d-8a8f-8df863d407d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:50.094786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6dbe20b0-ae04-4b1d-8a8f-8df863d407d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:50.102647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6dbe20b0-ae04-4b1d-8a8f-8df863d407d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:50.103202 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6dbe20b0-ae04-4b1d-8a8f-8df863d407d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:50.103871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6dbe20b0-ae04-4b1d-8a8f-8df863d407d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:50.104743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6dbe20b0-ae04-4b1d-8a8f-8df863d407d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:50.105252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6dbe20b0-ae04-4b1d-8a8f-8df863d407d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:50.105742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6dbe20b0-ae04-4b1d-8a8f-8df863d407d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:50.170390 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6dbe20b0-ae04-4b1d-8a8f-8df863d407d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.082355 Aug 07 01:22:50.171369 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 681/1324] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:50 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:50.462224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6c01d9ed-92e3-4f8f-b233-56fbe8de619b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:50.463323 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6c01d9ed-92e3-4f8f-b233-56fbe8de619b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:50.475425 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c01d9ed-92e3-4f8f-b233-56fbe8de619b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:50.476287 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c01d9ed-92e3-4f8f-b233-56fbe8de619b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:50.477071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c01d9ed-92e3-4f8f-b233-56fbe8de619b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:50.478359 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c01d9ed-92e3-4f8f-b233-56fbe8de619b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:50.479196 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c01d9ed-92e3-4f8f-b233-56fbe8de619b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:50.479981 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c01d9ed-92e3-4f8f-b233-56fbe8de619b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:50.572993 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6c01d9ed-92e3-4f8f-b233-56fbe8de619b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:50.583220 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6c01d9ed-92e3-4f8f-b233-56fbe8de619b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.128182 Aug 07 01:22:50.583853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 644/1325] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:50 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:51.184550 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4a657928-3c45-409f-969c-48a4f79e4617 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:51.188297 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4a657928-3c45-409f-969c-48a4f79e4617 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:51.195431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a657928-3c45-409f-969c-48a4f79e4617 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:51.195930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a657928-3c45-409f-969c-48a4f79e4617 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:51.196423 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a657928-3c45-409f-969c-48a4f79e4617 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:51.197157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a657928-3c45-409f-969c-48a4f79e4617 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:51.197620 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a657928-3c45-409f-969c-48a4f79e4617 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:51.198050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a657928-3c45-409f-969c-48a4f79e4617 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:51.231755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4a657928-3c45-409f-969c-48a4f79e4617 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051029 Aug 07 01:22:51.232225 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 682/1326] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:51 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:51.618636 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0b8164d7-bb17-40d2-8f8b-820099cf4383 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:51.619582 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0b8164d7-bb17-40d2-8f8b-820099cf4383 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:51.627050 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b8164d7-bb17-40d2-8f8b-820099cf4383 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:51.627548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b8164d7-bb17-40d2-8f8b-820099cf4383 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:51.627997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b8164d7-bb17-40d2-8f8b-820099cf4383 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:51.628833 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b8164d7-bb17-40d2-8f8b-820099cf4383 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:51.629300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b8164d7-bb17-40d2-8f8b-820099cf4383 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:51.629727 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0b8164d7-bb17-40d2-8f8b-820099cf4383 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:51.728709 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0b8164d7-bb17-40d2-8f8b-820099cf4383 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:51.741826 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0b8164d7-bb17-40d2-8f8b-820099cf4383 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.128056 Aug 07 01:22:51.742881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 645/1327] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:51 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:22:52.256390 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-47e91491-b3d3-4ad0-9129-97d122f4d457 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:52.264690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-47e91491-b3d3-4ad0-9129-97d122f4d457 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:52.280939 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-47e91491-b3d3-4ad0-9129-97d122f4d457 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:52.281850 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-47e91491-b3d3-4ad0-9129-97d122f4d457 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:52.282683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-47e91491-b3d3-4ad0-9129-97d122f4d457 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:52.284215 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-47e91491-b3d3-4ad0-9129-97d122f4d457 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:52.285264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-47e91491-b3d3-4ad0-9129-97d122f4d457 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:52.286748 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-47e91491-b3d3-4ad0-9129-97d122f4d457 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:52.351357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-47e91491-b3d3-4ad0-9129-97d122f4d457 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.106577 Aug 07 01:22:52.352579 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 683/1328] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:52 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:52.782900 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4d6560cc-fb10-4466-ace3-ae2df1650724 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:52.784048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4d6560cc-fb10-4466-ace3-ae2df1650724 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:52.796639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d6560cc-fb10-4466-ace3-ae2df1650724 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:52.797290 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d6560cc-fb10-4466-ace3-ae2df1650724 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:52.798073 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d6560cc-fb10-4466-ace3-ae2df1650724 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:52.799382 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d6560cc-fb10-4466-ace3-ae2df1650724 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:52.800224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d6560cc-fb10-4466-ace3-ae2df1650724 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:52.801136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d6560cc-fb10-4466-ace3-ae2df1650724 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:52.911058 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4d6560cc-fb10-4466-ace3-ae2df1650724 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:52.926712 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4d6560cc-fb10-4466-ace3-ae2df1650724 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.151316 Aug 07 01:22:52.927837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 646/1329] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:52 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:53.371276 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2fbdfdfc-a8ad-431c-b648-d43c46653a29 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:53.377743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2fbdfdfc-a8ad-431c-b648-d43c46653a29 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:53.387644 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fbdfdfc-a8ad-431c-b648-d43c46653a29 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:53.388269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fbdfdfc-a8ad-431c-b648-d43c46653a29 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:53.388763 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fbdfdfc-a8ad-431c-b648-d43c46653a29 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:53.389580 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fbdfdfc-a8ad-431c-b648-d43c46653a29 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:53.390060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fbdfdfc-a8ad-431c-b648-d43c46653a29 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:53.390551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fbdfdfc-a8ad-431c-b648-d43c46653a29 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:53.447786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2fbdfdfc-a8ad-431c-b648-d43c46653a29 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.082761 Aug 07 01:22:53.448818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 684/1330] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:53 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:53.963620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f70f6108-1fc6-4fcc-867c-40bbaa722482 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:53.964732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f70f6108-1fc6-4fcc-867c-40bbaa722482 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:53.977098 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f70f6108-1fc6-4fcc-867c-40bbaa722482 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:53.977601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f70f6108-1fc6-4fcc-867c-40bbaa722482 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:53.978061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f70f6108-1fc6-4fcc-867c-40bbaa722482 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:53.978788 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f70f6108-1fc6-4fcc-867c-40bbaa722482 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:53.979255 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f70f6108-1fc6-4fcc-867c-40bbaa722482 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:53.979685 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f70f6108-1fc6-4fcc-867c-40bbaa722482 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:54.062012 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f70f6108-1fc6-4fcc-867c-40bbaa722482 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:54.071688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f70f6108-1fc6-4fcc-867c-40bbaa722482 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.113322 Aug 07 01:22:54.072500 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 647/1331] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:53 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:54.462847 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dd786fe0-975d-457a-a4fd-c4b05a727f30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:54.465451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dd786fe0-975d-457a-a4fd-c4b05a727f30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:54.473383 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd786fe0-975d-457a-a4fd-c4b05a727f30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:54.473986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd786fe0-975d-457a-a4fd-c4b05a727f30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:54.474538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd786fe0-975d-457a-a4fd-c4b05a727f30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:54.475297 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd786fe0-975d-457a-a4fd-c4b05a727f30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:54.475770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd786fe0-975d-457a-a4fd-c4b05a727f30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:54.476212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd786fe0-975d-457a-a4fd-c4b05a727f30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:54.517539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dd786fe0-975d-457a-a4fd-c4b05a727f30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058927 Aug 07 01:22:54.518213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 685/1332] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:54 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:55.099500 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2e7a5a20-9a58-40ba-a9d6-f2f8cd3a0880 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:55.100705 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2e7a5a20-9a58-40ba-a9d6-f2f8cd3a0880 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:55.113159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e7a5a20-9a58-40ba-a9d6-f2f8cd3a0880 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:55.113974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e7a5a20-9a58-40ba-a9d6-f2f8cd3a0880 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:55.114753 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e7a5a20-9a58-40ba-a9d6-f2f8cd3a0880 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:55.116086 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e7a5a20-9a58-40ba-a9d6-f2f8cd3a0880 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:55.116960 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e7a5a20-9a58-40ba-a9d6-f2f8cd3a0880 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:55.117733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e7a5a20-9a58-40ba-a9d6-f2f8cd3a0880 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:55.220772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2e7a5a20-9a58-40ba-a9d6-f2f8cd3a0880 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:55.233139 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2e7a5a20-9a58-40ba-a9d6-f2f8cd3a0880 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.140764 Aug 07 01:22:55.234140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 648/1333] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:55 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:55.536099 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4dbe0cfb-66f4-494d-a36e-3853ddd18436 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:55.539957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4dbe0cfb-66f4-494d-a36e-3853ddd18436 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:55.552196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4dbe0cfb-66f4-494d-a36e-3853ddd18436 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:55.553121 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4dbe0cfb-66f4-494d-a36e-3853ddd18436 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:55.555665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4dbe0cfb-66f4-494d-a36e-3853ddd18436 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:55.555665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4dbe0cfb-66f4-494d-a36e-3853ddd18436 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:55.556155 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4dbe0cfb-66f4-494d-a36e-3853ddd18436 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:55.556562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4dbe0cfb-66f4-494d-a36e-3853ddd18436 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:55.616435 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4dbe0cfb-66f4-494d-a36e-3853ddd18436 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.088224 Aug 07 01:22:55.617488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 686/1334] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:55 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:56.268421 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3e9b8429-36b3-43f8-9ea8-d295c753769c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:56.269336 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3e9b8429-36b3-43f8-9ea8-d295c753769c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:56.277879 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3e9b8429-36b3-43f8-9ea8-d295c753769c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:56.278465 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3e9b8429-36b3-43f8-9ea8-d295c753769c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:56.278923 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3e9b8429-36b3-43f8-9ea8-d295c753769c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:56.279733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3e9b8429-36b3-43f8-9ea8-d295c753769c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:56.280345 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3e9b8429-36b3-43f8-9ea8-d295c753769c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:56.280814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3e9b8429-36b3-43f8-9ea8-d295c753769c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:56.390688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3e9b8429-36b3-43f8-9ea8-d295c753769c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:56.404907 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3e9b8429-36b3-43f8-9ea8-d295c753769c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.141583 Aug 07 01:22:56.406054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 649/1335] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:56 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:56.634654 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4d64711e-8128-4f56-8878-bb1e856dc73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:56.636806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4d64711e-8128-4f56-8878-bb1e856dc73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:56.644741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4d64711e-8128-4f56-8878-bb1e856dc73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:56.645236 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4d64711e-8128-4f56-8878-bb1e856dc73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:56.645681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4d64711e-8128-4f56-8878-bb1e856dc73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:56.646417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4d64711e-8128-4f56-8878-bb1e856dc73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:56.646875 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4d64711e-8128-4f56-8878-bb1e856dc73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:56.647303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4d64711e-8128-4f56-8878-bb1e856dc73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:56.683673 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4d64711e-8128-4f56-8878-bb1e856dc73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053605 Aug 07 01:22:56.684402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 687/1336] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:56 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:22:57.433481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ab50139c-a6c7-49eb-8aee-483d904c710a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:57.435383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ab50139c-a6c7-49eb-8aee-483d904c710a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:57.445831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab50139c-a6c7-49eb-8aee-483d904c710a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:57.446575 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab50139c-a6c7-49eb-8aee-483d904c710a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:57.447244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab50139c-a6c7-49eb-8aee-483d904c710a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:57.448432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab50139c-a6c7-49eb-8aee-483d904c710a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:57.449421 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab50139c-a6c7-49eb-8aee-483d904c710a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:57.450056 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab50139c-a6c7-49eb-8aee-483d904c710a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:57.559357 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ab50139c-a6c7-49eb-8aee-483d904c710a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:57.574916 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ab50139c-a6c7-49eb-8aee-483d904c710a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.147211 Aug 07 01:22:57.576010 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 650/1337] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:57 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:57.699737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d5b6f7bb-1fcd-4e18-8436-191cf79fa653 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:57.702589 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d5b6f7bb-1fcd-4e18-8436-191cf79fa653 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:57.713077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d5b6f7bb-1fcd-4e18-8436-191cf79fa653 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:57.713942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d5b6f7bb-1fcd-4e18-8436-191cf79fa653 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:57.714742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d5b6f7bb-1fcd-4e18-8436-191cf79fa653 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:57.716097 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d5b6f7bb-1fcd-4e18-8436-191cf79fa653 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:57.716959 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d5b6f7bb-1fcd-4e18-8436-191cf79fa653 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:57.717737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d5b6f7bb-1fcd-4e18-8436-191cf79fa653 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:57.776895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d5b6f7bb-1fcd-4e18-8436-191cf79fa653 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.082330 Aug 07 01:22:57.777838 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 688/1338] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:57 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:58.600386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0e1eb6a5-1bdd-4d53-b5f1-09095059ba09 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:58.601593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0e1eb6a5-1bdd-4d53-b5f1-09095059ba09 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:58.609392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e1eb6a5-1bdd-4d53-b5f1-09095059ba09 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:58.609903 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e1eb6a5-1bdd-4d53-b5f1-09095059ba09 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:58.610339 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e1eb6a5-1bdd-4d53-b5f1-09095059ba09 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:58.611887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e1eb6a5-1bdd-4d53-b5f1-09095059ba09 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:58.611887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e1eb6a5-1bdd-4d53-b5f1-09095059ba09 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:58.612377 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e1eb6a5-1bdd-4d53-b5f1-09095059ba09 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:58.719239 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0e1eb6a5-1bdd-4d53-b5f1-09095059ba09 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:58.729101 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0e1eb6a5-1bdd-4d53-b5f1-09095059ba09 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.135976 Aug 07 01:22:58.729791 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 651/1339] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:58 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:58.795508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2b381274-7411-4ad6-a7fb-6107880d75fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:58.799464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2b381274-7411-4ad6-a7fb-6107880d75fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:58.812755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b381274-7411-4ad6-a7fb-6107880d75fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:58.813795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b381274-7411-4ad6-a7fb-6107880d75fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:58.814647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b381274-7411-4ad6-a7fb-6107880d75fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:58.816017 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b381274-7411-4ad6-a7fb-6107880d75fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:58.816893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b381274-7411-4ad6-a7fb-6107880d75fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:58.817740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b381274-7411-4ad6-a7fb-6107880d75fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:58.883785 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2b381274-7411-4ad6-a7fb-6107880d75fb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.095374 Aug 07 01:22:58.884865 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 689/1340] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:58 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:22:59.767620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0d9068a1-97b7-4efb-a392-b40be3312c7c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:59.769050 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0d9068a1-97b7-4efb-a392-b40be3312c7c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:59.781428 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d9068a1-97b7-4efb-a392-b40be3312c7c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:59.782266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d9068a1-97b7-4efb-a392-b40be3312c7c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:59.783016 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d9068a1-97b7-4efb-a392-b40be3312c7c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:59.784328 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d9068a1-97b7-4efb-a392-b40be3312c7c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:59.785118 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d9068a1-97b7-4efb-a392-b40be3312c7c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:59.785803 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d9068a1-97b7-4efb-a392-b40be3312c7c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:59.899717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0d9068a1-97b7-4efb-a392-b40be3312c7c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:22:59.906894 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5d7989cc-463c-4053-80dc-ee7b684e919c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:22:59.911087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5d7989cc-463c-4053-80dc-ee7b684e919c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:22:59.911889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0d9068a1-97b7-4efb-a392-b40be3312c7c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.150922 Aug 07 01:22:59.916305 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 652/1341] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:22:59 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:22:59.931258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d7989cc-463c-4053-80dc-ee7b684e919c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:59.932586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d7989cc-463c-4053-80dc-ee7b684e919c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:59.933550 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d7989cc-463c-4053-80dc-ee7b684e919c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:59.935163 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d7989cc-463c-4053-80dc-ee7b684e919c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:22:59.936009 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d7989cc-463c-4053-80dc-ee7b684e919c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:22:59.939193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d7989cc-463c-4053-80dc-ee7b684e919c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:22:59.996833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5d7989cc-463c-4053-80dc-ee7b684e919c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.098213 Aug 07 01:22:59.997702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 690/1342] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:22:59 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:00.607542 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3c036833-fb69-4a75-b23f-6eb4c95b488f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:00.608470 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3c036833-fb69-4a75-b23f-6eb4c95b488f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:00.622997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3c036833-fb69-4a75-b23f-6eb4c95b488f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:00.624004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3c036833-fb69-4a75-b23f-6eb4c95b488f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:00.624863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3c036833-fb69-4a75-b23f-6eb4c95b488f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:00.626216 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3c036833-fb69-4a75-b23f-6eb4c95b488f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:00.627099 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3c036833-fb69-4a75-b23f-6eb4c95b488f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:00.627867 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3c036833-fb69-4a75-b23f-6eb4c95b488f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:00.676644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3c036833-fb69-4a75-b23f-6eb4c95b488f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Going to try to terminate instance {{(pid=117984) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:23:00.747623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.compute.api [None req-3c036833-fb69-4a75-b23f-6eb4c95b488f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Instance is already in deleting state, ignoring this request Aug 07 01:23:00.748844 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3c036833-fb69-4a75-b23f-6eb4c95b488f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "DELETE /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 204 len: 0 microversion: 2.1 time: 0.148864 Aug 07 01:23:00.749534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 653/1343] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:23:00 2022] DELETE /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 07 01:23:00.759262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9c329339-7dfa-43d3-a941-12316855df5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:00.759966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9c329339-7dfa-43d3-a941-12316855df5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:00.767444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9c329339-7dfa-43d3-a941-12316855df5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:00.767939 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9c329339-7dfa-43d3-a941-12316855df5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:00.768434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9c329339-7dfa-43d3-a941-12316855df5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:00.769176 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9c329339-7dfa-43d3-a941-12316855df5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:00.769638 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9c329339-7dfa-43d3-a941-12316855df5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:00.770106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9c329339-7dfa-43d3-a941-12316855df5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:00.865872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9c329339-7dfa-43d3-a941-12316855df5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:00.874600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9c329339-7dfa-43d3-a941-12316855df5a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.119937 Aug 07 01:23:00.875729 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 691/1344] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:00 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:01.013717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-af51bae6-2227-43d1-a7c9-059a3ee89ee0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:01.016293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-af51bae6-2227-43d1-a7c9-059a3ee89ee0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:01.023878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af51bae6-2227-43d1-a7c9-059a3ee89ee0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:01.024633 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af51bae6-2227-43d1-a7c9-059a3ee89ee0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:01.025202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af51bae6-2227-43d1-a7c9-059a3ee89ee0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:01.026082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af51bae6-2227-43d1-a7c9-059a3ee89ee0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:01.026657 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af51bae6-2227-43d1-a7c9-059a3ee89ee0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:01.027241 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-af51bae6-2227-43d1-a7c9-059a3ee89ee0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:01.047540 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:01.065778 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-af51bae6-2227-43d1-a7c9-059a3ee89ee0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057065 Aug 07 01:23:01.066495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 654/1345] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:23:01 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:01.897409 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ca61cda7-bff7-4280-8984-f5d640a7f9e6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:01.898604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ca61cda7-bff7-4280-8984-f5d640a7f9e6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:01.906432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ca61cda7-bff7-4280-8984-f5d640a7f9e6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:01.906926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ca61cda7-bff7-4280-8984-f5d640a7f9e6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:01.907397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ca61cda7-bff7-4280-8984-f5d640a7f9e6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:01.908130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ca61cda7-bff7-4280-8984-f5d640a7f9e6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:01.908707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ca61cda7-bff7-4280-8984-f5d640a7f9e6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:01.909084 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ca61cda7-bff7-4280-8984-f5d640a7f9e6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:01.987042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ca61cda7-bff7-4280-8984-f5d640a7f9e6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:01.995913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ca61cda7-bff7-4280-8984-f5d640a7f9e6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.103457 Aug 07 01:23:01.996710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 692/1346] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:01 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:02.079979 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-70b67ee5-2cc6-42fa-bf29-263cf1c31253 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:02.082294 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-70b67ee5-2cc6-42fa-bf29-263cf1c31253 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:02.090071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-70b67ee5-2cc6-42fa-bf29-263cf1c31253 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:02.090575 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-70b67ee5-2cc6-42fa-bf29-263cf1c31253 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:02.090989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-70b67ee5-2cc6-42fa-bf29-263cf1c31253 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:02.091729 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-70b67ee5-2cc6-42fa-bf29-263cf1c31253 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:02.092177 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-70b67ee5-2cc6-42fa-bf29-263cf1c31253 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:02.092703 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-70b67ee5-2cc6-42fa-bf29-263cf1c31253 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:02.128737 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-70b67ee5-2cc6-42fa-bf29-263cf1c31253 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053316 Aug 07 01:23:02.129403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 655/1347] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:23:02 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:03.018732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-056bafcd-16dd-4c45-9354-e5d16c2de774 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:03.019435 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-056bafcd-16dd-4c45-9354-e5d16c2de774 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:03.026889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-056bafcd-16dd-4c45-9354-e5d16c2de774 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:03.027430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-056bafcd-16dd-4c45-9354-e5d16c2de774 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:03.027908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-056bafcd-16dd-4c45-9354-e5d16c2de774 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:03.028813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-056bafcd-16dd-4c45-9354-e5d16c2de774 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:03.029323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-056bafcd-16dd-4c45-9354-e5d16c2de774 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:03.029795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-056bafcd-16dd-4c45-9354-e5d16c2de774 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:03.102904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-056bafcd-16dd-4c45-9354-e5d16c2de774 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:03.111794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-056bafcd-16dd-4c45-9354-e5d16c2de774 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.098404 Aug 07 01:23:03.112515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 693/1348] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:03 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:03.145777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7be24e59-0e6f-4c79-bc7c-fd3ae6e694e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:03.149051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7be24e59-0e6f-4c79-bc7c-fd3ae6e694e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:03.157140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7be24e59-0e6f-4c79-bc7c-fd3ae6e694e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:03.157732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7be24e59-0e6f-4c79-bc7c-fd3ae6e694e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:03.158215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7be24e59-0e6f-4c79-bc7c-fd3ae6e694e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:03.158972 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7be24e59-0e6f-4c79-bc7c-fd3ae6e694e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:03.159461 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7be24e59-0e6f-4c79-bc7c-fd3ae6e694e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:03.159919 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7be24e59-0e6f-4c79-bc7c-fd3ae6e694e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:03.193177 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7be24e59-0e6f-4c79-bc7c-fd3ae6e694e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053706 Aug 07 01:23:03.193890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 656/1349] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:23:03 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:23:04.073518 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5155c31f-6887-405d-920a-96a55204d469 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:04.074619 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5155c31f-6887-405d-920a-96a55204d469 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:04.079460 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=56 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:04.086080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5155c31f-6887-405d-920a-96a55204d469 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.086978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5155c31f-6887-405d-920a-96a55204d469 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.087576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5155c31f-6887-405d-920a-96a55204d469 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.088574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5155c31f-6887-405d-920a-96a55204d469 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.089189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5155c31f-6887-405d-920a-96a55204d469 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.089757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5155c31f-6887-405d-920a-96a55204d469 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.122621 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-57257bde-4591-4a0d-ad63-c265657dc068 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:04.123417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5155c31f-6887-405d-920a-96a55204d469 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Going to try to terminate instance {{(pid=117983) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:23:04.123564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-57257bde-4591-4a0d-ad63-c265657dc068 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:04.134602 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57257bde-4591-4a0d-ad63-c265657dc068 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.135317 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57257bde-4591-4a0d-ad63-c265657dc068 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.136068 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57257bde-4591-4a0d-ad63-c265657dc068 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.137373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57257bde-4591-4a0d-ad63-c265657dc068 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.138450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57257bde-4591-4a0d-ad63-c265657dc068 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.139445 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57257bde-4591-4a0d-ad63-c265657dc068 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.210949 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-57257bde-4591-4a0d-ad63-c265657dc068 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Going to try to terminate instance {{(pid=117984) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:23:04.247592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5155c31f-6887-405d-920a-96a55204d469 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "DELETE /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 204 len: 0 microversion: 2.20 time: 0.178038 Aug 07 01:23:04.248755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 694/1350] 10.222.0.26 () {60 vars in 1280 bytes} [Sun Aug 7 01:23:04 2022] DELETE /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 07 01:23:04.260730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5fc542bc-c62a-4bb4-9160-94dc6149867a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:04.262257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5fc542bc-c62a-4bb4-9160-94dc6149867a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:04.273961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fc542bc-c62a-4bb4-9160-94dc6149867a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.274862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fc542bc-c62a-4bb4-9160-94dc6149867a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.275701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fc542bc-c62a-4bb4-9160-94dc6149867a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.277132 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fc542bc-c62a-4bb4-9160-94dc6149867a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.278150 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fc542bc-c62a-4bb4-9160-94dc6149867a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.278803 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5fc542bc-c62a-4bb4-9160-94dc6149867a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.325409 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:04.383712 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-57257bde-4591-4a0d-ad63-c265657dc068 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 204 len: 0 microversion: 2.1 time: 0.267932 Aug 07 01:23:04.384443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 657/1351] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:23:04 2022] DELETE /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 07 01:23:04.387667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5fc542bc-c62a-4bb4-9160-94dc6149867a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:04.391347 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-62188dc2-0039-4ab0-b988-62a3d952dfce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:04.394648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-62188dc2-0039-4ab0-b988-62a3d952dfce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:04.402697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62188dc2-0039-4ab0-b988-62a3d952dfce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.403194 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62188dc2-0039-4ab0-b988-62a3d952dfce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.403422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5fc542bc-c62a-4bb4-9160-94dc6149867a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.151016 Aug 07 01:23:04.403846 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62188dc2-0039-4ab0-b988-62a3d952dfce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.404942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62188dc2-0039-4ab0-b988-62a3d952dfce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.405367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 695/1352] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:04 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:04.405552 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62188dc2-0039-4ab0-b988-62a3d952dfce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.406057 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62188dc2-0039-4ab0-b988-62a3d952dfce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.414515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f9d914ea-a4c7-4d03-80ca-ef5bb5935926 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:04.415163 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f9d914ea-a4c7-4d03-80ca-ef5bb5935926 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:04.426347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9d914ea-a4c7-4d03-80ca-ef5bb5935926 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.427112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9d914ea-a4c7-4d03-80ca-ef5bb5935926 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.428144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9d914ea-a4c7-4d03-80ca-ef5bb5935926 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.429366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9d914ea-a4c7-4d03-80ca-ef5bb5935926 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.430194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9d914ea-a4c7-4d03-80ca-ef5bb5935926 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.430935 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f9d914ea-a4c7-4d03-80ca-ef5bb5935926 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.447816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-62188dc2-0039-4ab0-b988-62a3d952dfce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062081 Aug 07 01:23:04.448636 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 658/1353] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:23:04 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:04.457977 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c7be3fee-cb75-40d1-9094-90ccc1c38d47 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:04.458625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c7be3fee-cb75-40d1-9094-90ccc1c38d47 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:04.465228 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7be3fee-cb75-40d1-9094-90ccc1c38d47 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.465720 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7be3fee-cb75-40d1-9094-90ccc1c38d47 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.466160 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7be3fee-cb75-40d1-9094-90ccc1c38d47 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.466894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7be3fee-cb75-40d1-9094-90ccc1c38d47 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.467352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7be3fee-cb75-40d1-9094-90ccc1c38d47 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.467773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7be3fee-cb75-40d1-9094-90ccc1c38d47 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.482783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f9d914ea-a4c7-4d03-80ca-ef5bb5935926 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Going to try to terminate instance {{(pid=117983) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:23:04.612864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f9d914ea-a4c7-4d03-80ca-ef5bb5935926 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "DELETE /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 204 len: 0 microversion: 2.1 time: 0.205173 Aug 07 01:23:04.613661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 696/1354] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:23:04 2022] DELETE /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 07 01:23:04.622131 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e34fb04a-066a-4651-b44c-67e3b112804a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:04.622857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e34fb04a-066a-4651-b44c-67e3b112804a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:04.629931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e34fb04a-066a-4651-b44c-67e3b112804a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.630421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e34fb04a-066a-4651-b44c-67e3b112804a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.630869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e34fb04a-066a-4651-b44c-67e3b112804a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.631690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e34fb04a-066a-4651-b44c-67e3b112804a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.632255 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e34fb04a-066a-4651-b44c-67e3b112804a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.632774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e34fb04a-066a-4651-b44c-67e3b112804a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.811535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e34fb04a-066a-4651-b44c-67e3b112804a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:04.821305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e34fb04a-066a-4651-b44c-67e3b112804a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:04.823049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e34fb04a-066a-4651-b44c-67e3b112804a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:04.824815 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e34fb04a-066a-4651-b44c-67e3b112804a tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1725 microversion: 2.20 time: 0.209246 Aug 07 01:23:04.825457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 697/1355] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:23:04 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1725 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:23:04.835114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1886db3d-b642-4303-baf7-c63a9b97688e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:04.836344 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1886db3d-b642-4303-baf7-c63a9b97688e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:04.848426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1886db3d-b642-4303-baf7-c63a9b97688e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.849365 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1886db3d-b642-4303-baf7-c63a9b97688e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.850202 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1886db3d-b642-4303-baf7-c63a9b97688e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:04.851698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1886db3d-b642-4303-baf7-c63a9b97688e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:04.852716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1886db3d-b642-4303-baf7-c63a9b97688e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:04.853539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1886db3d-b642-4303-baf7-c63a9b97688e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:05.055100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1886db3d-b642-4303-baf7-c63a9b97688e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:05.064656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1886db3d-b642-4303-baf7-c63a9b97688e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1683 microversion: 2.1 time: 0.237422 Aug 07 01:23:05.065249 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 698/1356] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:04 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1683 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:05.071631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3ea968a4-1696-4948-b3f1-bc93332372aa tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:05.072633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3ea968a4-1696-4948-b3f1-bc93332372aa tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:05.081530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ea968a4-1696-4948-b3f1-bc93332372aa tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:05.082597 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ea968a4-1696-4948-b3f1-bc93332372aa tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:05.082597 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ea968a4-1696-4948-b3f1-bc93332372aa tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:05.083161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ea968a4-1696-4948-b3f1-bc93332372aa tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:05.083613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ea968a4-1696-4948-b3f1-bc93332372aa tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:05.084040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ea968a4-1696-4948-b3f1-bc93332372aa tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:05.280830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3ea968a4-1696-4948-b3f1-bc93332372aa tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:05.289311 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3ea968a4-1696-4948-b3f1-bc93332372aa tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1673 microversion: 2.1 time: 0.222848 Aug 07 01:23:05.290002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 699/1357] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:05 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1673 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:05.452517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a6ccd915-cbde-4a5d-b751-8887b64a3f5d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:05.453746 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a6ccd915-cbde-4a5d-b751-8887b64a3f5d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:05.467572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6ccd915-cbde-4a5d-b751-8887b64a3f5d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:05.470681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6ccd915-cbde-4a5d-b751-8887b64a3f5d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:05.471593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6ccd915-cbde-4a5d-b751-8887b64a3f5d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:05.472946 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6ccd915-cbde-4a5d-b751-8887b64a3f5d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:05.473817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6ccd915-cbde-4a5d-b751-8887b64a3f5d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:05.474599 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6ccd915-cbde-4a5d-b751-8887b64a3f5d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:05.590700 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a6ccd915-cbde-4a5d-b751-8887b64a3f5d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:05.607003 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a6ccd915-cbde-4a5d-b751-8887b64a3f5d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.178881 Aug 07 01:23:05.612948 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 700/1358] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:05 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:05.619345 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-727258dc-6758-44f3-8d14-eec4866818c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:05.624142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-727258dc-6758-44f3-8d14-eec4866818c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:05.635528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-727258dc-6758-44f3-8d14-eec4866818c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:05.636730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-727258dc-6758-44f3-8d14-eec4866818c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:05.637664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-727258dc-6758-44f3-8d14-eec4866818c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:05.639343 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-727258dc-6758-44f3-8d14-eec4866818c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:05.640385 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-727258dc-6758-44f3-8d14-eec4866818c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:05.641257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-727258dc-6758-44f3-8d14-eec4866818c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:05.692084 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-727258dc-6758-44f3-8d14-eec4866818c0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080542 Aug 07 01:23:05.693118 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 701/1359] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:23:05 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:05.867251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1672bf7c-5cb5-45a2-b50c-4aae4c0abf7d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:05.867927 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1672bf7c-5cb5-45a2-b50c-4aae4c0abf7d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:05.875565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1672bf7c-5cb5-45a2-b50c-4aae4c0abf7d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:05.876186 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1672bf7c-5cb5-45a2-b50c-4aae4c0abf7d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:05.876692 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1672bf7c-5cb5-45a2-b50c-4aae4c0abf7d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:05.877622 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1672bf7c-5cb5-45a2-b50c-4aae4c0abf7d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:05.878207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1672bf7c-5cb5-45a2-b50c-4aae4c0abf7d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:05.879497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1672bf7c-5cb5-45a2-b50c-4aae4c0abf7d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:06.268930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1672bf7c-5cb5-45a2-b50c-4aae4c0abf7d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:06.292504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1672bf7c-5cb5-45a2-b50c-4aae4c0abf7d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:06.297214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1672bf7c-5cb5-45a2-b50c-4aae4c0abf7d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:06.301053 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1672bf7c-5cb5-45a2-b50c-4aae4c0abf7d tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1725 microversion: 2.20 time: 0.437172 Aug 07 01:23:06.317000 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 702/1360] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:23:05 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1725 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 07 01:23:06.332473 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-caa070f6-0236-4634-83ba-6c1e04c8cc94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:06.334094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-caa070f6-0236-4634-83ba-6c1e04c8cc94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:06.347918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caa070f6-0236-4634-83ba-6c1e04c8cc94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:06.349299 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caa070f6-0236-4634-83ba-6c1e04c8cc94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:06.351314 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caa070f6-0236-4634-83ba-6c1e04c8cc94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:06.352193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caa070f6-0236-4634-83ba-6c1e04c8cc94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:06.353080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caa070f6-0236-4634-83ba-6c1e04c8cc94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:06.354627 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-caa070f6-0236-4634-83ba-6c1e04c8cc94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:06.678873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-caa070f6-0236-4634-83ba-6c1e04c8cc94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:06.707386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-caa070f6-0236-4634-83ba-6c1e04c8cc94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1683 microversion: 2.1 time: 0.387776 Aug 07 01:23:06.708600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 703/1361] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:06 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1683 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:06.718535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6cc5679f-ee61-42d4-800c-9e4da51c8afd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:06.720146 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6cc5679f-ee61-42d4-800c-9e4da51c8afd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:06.732511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6cc5679f-ee61-42d4-800c-9e4da51c8afd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:06.734539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6cc5679f-ee61-42d4-800c-9e4da51c8afd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:06.735892 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6cc5679f-ee61-42d4-800c-9e4da51c8afd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:06.738262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6cc5679f-ee61-42d4-800c-9e4da51c8afd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:06.738970 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6cc5679f-ee61-42d4-800c-9e4da51c8afd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:06.739591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6cc5679f-ee61-42d4-800c-9e4da51c8afd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:07.109599 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6cc5679f-ee61-42d4-800c-9e4da51c8afd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:07.125228 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6cc5679f-ee61-42d4-800c-9e4da51c8afd tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1673 microversion: 2.1 time: 0.413860 Aug 07 01:23:07.127486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 704/1362] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:06 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1673 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:07.140710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e32ceb77-09df-4176-ac7a-7deafc835441 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:07.145147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e32ceb77-09df-4176-ac7a-7deafc835441 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:07.168805 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e32ceb77-09df-4176-ac7a-7deafc835441 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:07.170515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e32ceb77-09df-4176-ac7a-7deafc835441 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:07.171682 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e32ceb77-09df-4176-ac7a-7deafc835441 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:07.173628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e32ceb77-09df-4176-ac7a-7deafc835441 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:07.174485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e32ceb77-09df-4176-ac7a-7deafc835441 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:07.175270 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e32ceb77-09df-4176-ac7a-7deafc835441 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:07.328970 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e32ceb77-09df-4176-ac7a-7deafc835441 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:07.368712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e32ceb77-09df-4176-ac7a-7deafc835441 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.238225 Aug 07 01:23:07.371074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 705/1363] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:07 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:07.380809 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-100be347-eec5-4b7e-ab08-c1f1f02d3904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:07.385350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-100be347-eec5-4b7e-ab08-c1f1f02d3904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:07.400065 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-100be347-eec5-4b7e-ab08-c1f1f02d3904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:07.401412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-100be347-eec5-4b7e-ab08-c1f1f02d3904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:07.409518 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-100be347-eec5-4b7e-ab08-c1f1f02d3904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:07.414183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-100be347-eec5-4b7e-ab08-c1f1f02d3904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:07.415326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-100be347-eec5-4b7e-ab08-c1f1f02d3904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:07.416408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-100be347-eec5-4b7e-ab08-c1f1f02d3904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:07.477854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-100be347-eec5-4b7e-ab08-c1f1f02d3904 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.104534 Aug 07 01:23:07.479280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 706/1364] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:23:07 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:07.591934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-cee25706-a215-4a02-bede-878c25214b32 req-f1e27633-1fa3-4bf8-9945-0969374478a0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "00b4795f-ad25-4565-bc64-10cbae94641a", "name": "network-vif-deleted", "tag": "b2a8f175-ffc8-4e6e-a3f6-a939be50f0c9"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:23:07.621503 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-cee25706-a215-4a02-bede-878c25214b32 req-f1e27633-1fa3-4bf8-9945-0969374478a0 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:07.622571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-cee25706-a215-4a02-bede-878c25214b32 req-f1e27633-1fa3-4bf8-9945-0969374478a0 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:07.623696 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-cee25706-a215-4a02-bede-878c25214b32 req-f1e27633-1fa3-4bf8-9945-0969374478a0 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:07.641810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-cee25706-a215-4a02-bede-878c25214b32 req-f1e27633-1fa3-4bf8-9945-0969374478a0 service nova] Creating event network-vif-deleted:b2a8f175-ffc8-4e6e-a3f6-a939be50f0c9 for instance 00b4795f-ad25-4565-bc64-10cbae94641a on hv-cinder-maste Aug 07 01:23:07.650805 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-cee25706-a215-4a02-bede-878c25214b32 req-f1e27633-1fa3-4bf8-9945-0969374478a0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.166857 Aug 07 01:23:07.652040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 707/1365] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:23:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:07.660707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:07.661393 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:07.671704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:07.672224 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:07.672706 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:07.673562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:07.674044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:07.674684 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:07.744058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:07.745005 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:07.745649 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:07.746378 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.objects.instance [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lazy-loading 'fault' on Instance uuid 00b4795f-ad25-4565-bc64-10cbae94641a {{(pid=117983) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:23:07.839145 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:07.868258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:07.879299 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '5fafe42957914b929f9ffee53ab6e7ac', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1b1d33b8c71e41e6bf23d4ad8590d2c2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:07.880762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c2cdd0f8-a71e-4fc4-9fce-b6aa0b58dcce tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 200 len: 1371 microversion: 2.20 time: 0.227039 Aug 07 01:23:07.885311 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 708/1366] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:23:07 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 1371 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 07 01:23:07.892490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-203737f4-6a0c-43b8-9f13-20b489153fa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:07.900807 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-203737f4-6a0c-43b8-9f13-20b489153fa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:07.928575 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-203737f4-6a0c-43b8-9f13-20b489153fa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:07.929085 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-203737f4-6a0c-43b8-9f13-20b489153fa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:07.929541 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-203737f4-6a0c-43b8-9f13-20b489153fa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:07.935730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-203737f4-6a0c-43b8-9f13-20b489153fa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:07.941421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-203737f4-6a0c-43b8-9f13-20b489153fa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:07.942995 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-203737f4-6a0c-43b8-9f13-20b489153fa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:08.045689 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-203737f4-6a0c-43b8-9f13-20b489153fa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:08.059014 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-203737f4-6a0c-43b8-9f13-20b489153fa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1616 microversion: 2.1 time: 0.171952 Aug 07 01:23:08.060402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 709/1367] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:07 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1616 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:08.165805 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-b152cf8d-1c67-4a3c-909e-f1c135b24ac6 req-9d93ed5d-8996-4a37-898b-8ccacba122f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e86efc1a-b689-454a-8683-526a9c18e222", "name": "network-vif-deleted", "tag": "f5f83050-c635-4216-9f73-3f7cd60024ff"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:23:08.191358 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-b152cf8d-1c67-4a3c-909e-f1c135b24ac6 req-9d93ed5d-8996-4a37-898b-8ccacba122f4 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:08.192537 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-b152cf8d-1c67-4a3c-909e-f1c135b24ac6 req-9d93ed5d-8996-4a37-898b-8ccacba122f4 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:08.195578 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-b152cf8d-1c67-4a3c-909e-f1c135b24ac6 req-9d93ed5d-8996-4a37-898b-8ccacba122f4 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:08.216160 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-b152cf8d-1c67-4a3c-909e-f1c135b24ac6 req-9d93ed5d-8996-4a37-898b-8ccacba122f4 service nova] Creating event network-vif-deleted:f5f83050-c635-4216-9f73-3f7cd60024ff for instance e86efc1a-b689-454a-8683-526a9c18e222 on hv-cinder-maste Aug 07 01:23:08.222777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-b152cf8d-1c67-4a3c-909e-f1c135b24ac6 req-9d93ed5d-8996-4a37-898b-8ccacba122f4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.160651 Aug 07 01:23:08.224211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 710/1368] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:23:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:08.232807 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d003176a-c3f6-4e29-81f2-6c4bf205c2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:08.235492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d003176a-c3f6-4e29-81f2-6c4bf205c2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:08.247307 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d003176a-c3f6-4e29-81f2-6c4bf205c2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:08.248487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d003176a-c3f6-4e29-81f2-6c4bf205c2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:08.249353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d003176a-c3f6-4e29-81f2-6c4bf205c2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:08.251274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d003176a-c3f6-4e29-81f2-6c4bf205c2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:08.251998 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d003176a-c3f6-4e29-81f2-6c4bf205c2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:08.252858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d003176a-c3f6-4e29-81f2-6c4bf205c2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:08.288890 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:08.373221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d003176a-c3f6-4e29-81f2-6c4bf205c2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ae9c7ef17b746fc8f87e3c22a241b76', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fd9ab6d98a44605a57bd8ae9535a686', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:08.385668 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d003176a-c3f6-4e29-81f2-6c4bf205c2a9 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 200 len: 1322 microversion: 2.1 time: 0.158736 Aug 07 01:23:08.386672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 711/1369] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:08 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 1322 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:08.402286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1b04ebc9-e00e-4e71-8fa6-489912fe84e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:08.403439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1b04ebc9-e00e-4e71-8fa6-489912fe84e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:08.413953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b04ebc9-e00e-4e71-8fa6-489912fe84e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:08.414642 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b04ebc9-e00e-4e71-8fa6-489912fe84e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:08.415332 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b04ebc9-e00e-4e71-8fa6-489912fe84e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:08.416454 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b04ebc9-e00e-4e71-8fa6-489912fe84e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:08.416958 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b04ebc9-e00e-4e71-8fa6-489912fe84e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:08.417405 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b04ebc9-e00e-4e71-8fa6-489912fe84e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:08.528592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1b04ebc9-e00e-4e71-8fa6-489912fe84e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:08.543161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1b04ebc9-e00e-4e71-8fa6-489912fe84e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.144552 Aug 07 01:23:08.555900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 712/1370] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:08 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:08.555900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-92d57344-a72b-45fc-bfd9-241874411cd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:08.558826 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-92d57344-a72b-45fc-bfd9-241874411cd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:08.570799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-92d57344-a72b-45fc-bfd9-241874411cd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:08.571652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-92d57344-a72b-45fc-bfd9-241874411cd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:08.572461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-92d57344-a72b-45fc-bfd9-241874411cd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:08.573784 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-92d57344-a72b-45fc-bfd9-241874411cd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:08.574864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-92d57344-a72b-45fc-bfd9-241874411cd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:08.575722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-92d57344-a72b-45fc-bfd9-241874411cd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:08.639187 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-92d57344-a72b-45fc-bfd9-241874411cd7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.092545 Aug 07 01:23:08.640384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 713/1371] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:23:08 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:08.937232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c2755686-2272-4f70-97d7-d4cd2aaf9471 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:08.940138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c2755686-2272-4f70-97d7-d4cd2aaf9471 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:08.952357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2755686-2272-4f70-97d7-d4cd2aaf9471 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:08.954268 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2755686-2272-4f70-97d7-d4cd2aaf9471 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:08.955518 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2755686-2272-4f70-97d7-d4cd2aaf9471 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:08.957238 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2755686-2272-4f70-97d7-d4cd2aaf9471 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:08.958235 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2755686-2272-4f70-97d7-d4cd2aaf9471 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:08.959127 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c2755686-2272-4f70-97d7-d4cd2aaf9471 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:08.985670 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-c2755686-2272-4f70-97d7-d4cd2aaf9471 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] HTTP exception thrown: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c2755686-2272-4f70-97d7-d4cd2aaf9471 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Request method failure captured: Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a HTTP/1.1 Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xDuZbhbLDbrOLrZVY9vlm_HtWVxIubBKHAStAP1FnjiA_U5zjLpQJv9uy1CLGRNLdulE-dSDkxwztkKA7ofKoolemIQ0OeMpjkkOauZx4Jr-U23jKoErXsQ2oRdJqqP6y71FTJ9znyT4ExNnLWW_EBEi4UJWyaqL-JEhBPxpiLjkk0 Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Openstack-Nova-Api-Version: 2.20 Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 1b1d33b8c71e41e6bf23d4ad8590d2c2 Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:23:08.988290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:23:08.989993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:23:08.989993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:23:08.989993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 1b1d33b8c71e41e6bf23d4ad8590d2c2 Aug 07 01:23:08.989993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:23:08.989993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-AttachVolumeShelveTestJSON-734115863-project Aug 07 01:23:08.989993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:23:08.989993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:23:08.989993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 5fafe42957914b929f9ffee53ab6e7ac Aug 07 01:23:08.989993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-AttachVolumeShelveTestJSON-734115863-project Aug 07 01:23:08.989993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:23:08.989993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:23:08.989993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '00b4795f-ad25-4565-bc64-10cbae94641a'} Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:23:08.990775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:23:08.993032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:08.997353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c2755686-2272-4f70-97d7-d4cd2aaf9471 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Returning 404 to user: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:23:08.998793 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c2755686-2272-4f70-97d7-d4cd2aaf9471 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 404 len: 111 microversion: 2.20 time: 0.069459 Aug 07 01:23:09.000145 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 714/1372] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:23:08 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 07 01:23:09.014408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-84aa1883-204e-4028-86dd-a261946ab03c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '.version_select at 0x7f1fad458d30>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:09.053272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-84aa1883-204e-4028-86dd-a261946ab03c tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-672649657" status: 204 len: 0 microversion: 2.20 time: 0.045507 Aug 07 01:23:09.054436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 715/1373] 10.222.0.26 () {60 vars in 1259 bytes} [Sun Aug 7 01:23:09 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-672649657 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Aug 07 01:23:09.097278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4b11a68b-50bf-47ae-877d-5ecb1dfbd1ec tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:09.097917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4b11a68b-50bf-47ae-877d-5ecb1dfbd1ec tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:09.111723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b11a68b-50bf-47ae-877d-5ecb1dfbd1ec tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:09.112742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b11a68b-50bf-47ae-877d-5ecb1dfbd1ec tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:09.113207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b11a68b-50bf-47ae-877d-5ecb1dfbd1ec tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:09.113986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b11a68b-50bf-47ae-877d-5ecb1dfbd1ec tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:09.117313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b11a68b-50bf-47ae-877d-5ecb1dfbd1ec tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:09.121725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b11a68b-50bf-47ae-877d-5ecb1dfbd1ec tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:09.213429 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b11a68b-50bf-47ae-877d-5ecb1dfbd1ec tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:09.214174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b11a68b-50bf-47ae-877d-5ecb1dfbd1ec tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:09.216150 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b11a68b-50bf-47ae-877d-5ecb1dfbd1ec tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:09.216150 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.objects.instance [None req-4b11a68b-50bf-47ae-877d-5ecb1dfbd1ec tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lazy-loading 'fault' on Instance uuid e86efc1a-b689-454a-8683-526a9c18e222 {{(pid=117983) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:23:09.273046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4b11a68b-50bf-47ae-877d-5ecb1dfbd1ec tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:09.286992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4b11a68b-50bf-47ae-877d-5ecb1dfbd1ec tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 200 len: 1332 microversion: 2.1 time: 0.196291 Aug 07 01:23:09.301188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 716/1374] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:09 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 1332 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:09.431885 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5a57333a-c4dd-4b50-ad55-c67a56a0c4f4 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:09.442150 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5a57333a-c4dd-4b50-ad55-c67a56a0c4f4 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:09.477151 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5a57333a-c4dd-4b50-ad55-c67a56a0c4f4 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:09.477655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5a57333a-c4dd-4b50-ad55-c67a56a0c4f4 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:09.478101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5a57333a-c4dd-4b50-ad55-c67a56a0c4f4 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:09.487605 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5a57333a-c4dd-4b50-ad55-c67a56a0c4f4 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:09.488124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5a57333a-c4dd-4b50-ad55-c67a56a0c4f4 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:09.488611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5a57333a-c4dd-4b50-ad55-c67a56a0c4f4 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:09.521045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-5a57333a-c4dd-4b50-ad55-c67a56a0c4f4 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] HTTP exception thrown: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5a57333a-c4dd-4b50-ad55-c67a56a0c4f4 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Request method failure captured: Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a HTTP/1.1 Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xD9cznP8hWf4pmxdU3hNl0ZKgrnQCZpWvAOumCUzV3uHmYQXZPbB6q2YtbZ5VRmM0fulvc4-ky89A5e1E7XmdwEvMsFQ8K0SJLQxjef3QqDDgPjeGMqBD9ErxCwISYFPrZdE0TGsrQI6E1EI8ML9K90t4_XH8zTJg5NaAH5XIw5714 Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 7fd9ab6d98a44605a57bd8ae9535a686 Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:23:09.522796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:23:09.525738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:23:09.525738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:23:09.525738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 7fd9ab6d98a44605a57bd8ae9535a686 Aug 07 01:23:09.525738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:23:09.525738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-AttachVolumeTestJSON-1659395094-project Aug 07 01:23:09.525738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:23:09.525738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:23:09.525738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 4ae9c7ef17b746fc8f87e3c22a241b76 Aug 07 01:23:09.525738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-AttachVolumeTestJSON-1659395094-project Aug 07 01:23:09.525738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:23:09.525738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:23:09.525738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '8e8c48f1-19d6-4691-9e54-e5d84962485a'} Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:23:09.527981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:23:09.532930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:09.537350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5a57333a-c4dd-4b50-ad55-c67a56a0c4f4 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Returning 404 to user: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:23:09.537350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5a57333a-c4dd-4b50-ad55-c67a56a0c4f4 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 404 len: 111 microversion: 2.1 time: 0.102818 Aug 07 01:23:09.537350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 717/1375] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:09 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 111 bytes in 113 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 07 01:23:09.544109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c3c2f825-4762-445b-a5fd-cf517a841582 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '.version_select at 0x7f1fad5cb310>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:09.595114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c3c2f825-4762-445b-a5fd-cf517a841582 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2048332820" status: 202 len: 0 microversion: 2.1 time: 0.057952 Aug 07 01:23:09.597087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 718/1376] 10.222.0.26 () {58 vars in 1221 bytes} [Sun Aug 7 01:23:09 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2048332820 => generated 0 bytes in 65 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:23:09.632504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3c537ce5-afc7-4f53-a448-cb78ed7a7fe9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:09.637838 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3c537ce5-afc7-4f53-a448-cb78ed7a7fe9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:09.678535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c537ce5-afc7-4f53-a448-cb78ed7a7fe9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:09.679980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c537ce5-afc7-4f53-a448-cb78ed7a7fe9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:09.681261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c537ce5-afc7-4f53-a448-cb78ed7a7fe9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:09.682776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c537ce5-afc7-4f53-a448-cb78ed7a7fe9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:09.683736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c537ce5-afc7-4f53-a448-cb78ed7a7fe9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:09.684896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c537ce5-afc7-4f53-a448-cb78ed7a7fe9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:09.961389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3c537ce5-afc7-4f53-a448-cb78ed7a7fe9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:09.979294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3c537ce5-afc7-4f53-a448-cb78ed7a7fe9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.379795 Aug 07 01:23:09.980306 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 719/1377] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:09 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:10.006819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0d63d601-1fa1-4b08-a78e-ae2ae51fe027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:10.010863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0d63d601-1fa1-4b08-a78e-ae2ae51fe027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:10.020775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d63d601-1fa1-4b08-a78e-ae2ae51fe027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:10.031283 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d63d601-1fa1-4b08-a78e-ae2ae51fe027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:10.032075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d63d601-1fa1-4b08-a78e-ae2ae51fe027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:10.033430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d63d601-1fa1-4b08-a78e-ae2ae51fe027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:10.034029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d63d601-1fa1-4b08-a78e-ae2ae51fe027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:10.034553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d63d601-1fa1-4b08-a78e-ae2ae51fe027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:10.155555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0d63d601-1fa1-4b08-a78e-ae2ae51fe027 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.157313 Aug 07 01:23:10.160895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 720/1378] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:23:09 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:10.256912 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-e7c59980-f005-4400-9a11-1061b96d0587 req-e2a33b9a-13df-49f5-b424-6ff89e450564 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e8c48f1-19d6-4691-9e54-e5d84962485a", "name": "network-vif-deleted", "tag": "cca96b0c-1343-4c42-8e76-9d5bf2321cee"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:23:10.285252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-e7c59980-f005-4400-9a11-1061b96d0587 req-e2a33b9a-13df-49f5-b424-6ff89e450564 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:10.287216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-e7c59980-f005-4400-9a11-1061b96d0587 req-e2a33b9a-13df-49f5-b424-6ff89e450564 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:10.290027 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-e7c59980-f005-4400-9a11-1061b96d0587 req-e2a33b9a-13df-49f5-b424-6ff89e450564 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:10.321031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-e7c59980-f005-4400-9a11-1061b96d0587 req-e2a33b9a-13df-49f5-b424-6ff89e450564 service nova] Creating event network-vif-deleted:cca96b0c-1343-4c42-8e76-9d5bf2321cee for instance 8e8c48f1-19d6-4691-9e54-e5d84962485a on hv-cinder-maste Aug 07 01:23:10.331122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-e7c59980-f005-4400-9a11-1061b96d0587 req-e2a33b9a-13df-49f5-b424-6ff89e450564 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082660 Aug 07 01:23:10.333124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 721/1379] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:23:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:10.347949 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-75ec0a8b-19bb-42f4-bb13-bdaab5e63850 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:10.349522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-75ec0a8b-19bb-42f4-bb13-bdaab5e63850 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:10.364721 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-75ec0a8b-19bb-42f4-bb13-bdaab5e63850 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:10.365061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-75ec0a8b-19bb-42f4-bb13-bdaab5e63850 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:10.365926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-75ec0a8b-19bb-42f4-bb13-bdaab5e63850 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:10.367530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-75ec0a8b-19bb-42f4-bb13-bdaab5e63850 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:10.368509 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-75ec0a8b-19bb-42f4-bb13-bdaab5e63850 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:10.369428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-75ec0a8b-19bb-42f4-bb13-bdaab5e63850 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:10.393334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-75ec0a8b-19bb-42f4-bb13-bdaab5e63850 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] HTTP exception thrown: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-75ec0a8b-19bb-42f4-bb13-bdaab5e63850 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Request method failure captured: Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 HTTP/1.1 Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xD021Vt2D0N359UOKX9xymFYpnZ79FQqLbXwvg5UbTRYuTTlR_HceEoHaOg5fvzy7nSB6FZNa7SVaU3q4lxIRVTXMupRK2mSDBaPiXGNUdpidSOmvxEoLqM4Zox-kqkoV9_K0_oxsudsRHb6jwecdh1rLe0262HFBUWOJmPZoVity8 Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:23:10.395828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:23:10.398113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:23:10.398113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:23:10.398113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:23:10.398113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:23:10.398113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:23:10.398113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:23:10.398113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:23:10.398113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: e65f0f1ba54f483abff788974b1256cb Aug 07 01:23:10.398113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:23:10.398113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:23:10.398113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:23:10.398113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': 'e86efc1a-b689-454a-8683-526a9c18e222'} Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:23:10.399258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:23:10.401109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:23:10.404206 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-75ec0a8b-19bb-42f4-bb13-bdaab5e63850 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Returning 404 to user: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:23:10.406039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-75ec0a8b-19bb-42f4-bb13-bdaab5e63850 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 404 len: 111 microversion: 2.1 time: 0.070348 Aug 07 01:23:10.407494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 722/1380] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:10 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 111 bytes in 74 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:23:10.430057 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-29d95166-a4dc-4b97-b709-6b1027149253 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '.version_select at 0x7f1fad5cbf70>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:10.457251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-29d95166-a4dc-4b97-b709-6b1027149253 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1298711506" status: 202 len: 0 microversion: 2.1 time: 0.039000 Aug 07 01:23:10.462507 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 723/1381] 10.222.0.26 () {58 vars in 1221 bytes} [Sun Aug 7 01:23:10 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1298711506 => generated 0 bytes in 44 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:23:11.007400 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-583fd46b-fe95-43a0-8281-789bbd989fc2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:11.008629 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-583fd46b-fe95-43a0-8281-789bbd989fc2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:11.038236 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-583fd46b-fe95-43a0-8281-789bbd989fc2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:11.049666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-583fd46b-fe95-43a0-8281-789bbd989fc2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:11.051298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-583fd46b-fe95-43a0-8281-789bbd989fc2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:11.053116 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-583fd46b-fe95-43a0-8281-789bbd989fc2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:11.054092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-583fd46b-fe95-43a0-8281-789bbd989fc2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:11.054914 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-583fd46b-fe95-43a0-8281-789bbd989fc2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:11.167208 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-583fd46b-fe95-43a0-8281-789bbd989fc2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:11.177157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-583fd46b-fe95-43a0-8281-789bbd989fc2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.177643 Aug 07 01:23:11.178090 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 724/1382] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:10 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:11.184198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7ae66db9-c129-4a17-bd99-9284ae2e40ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:11.187283 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7ae66db9-c129-4a17-bd99-9284ae2e40ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:11.198525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ae66db9-c129-4a17-bd99-9284ae2e40ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:11.199034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ae66db9-c129-4a17-bd99-9284ae2e40ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:11.199855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ae66db9-c129-4a17-bd99-9284ae2e40ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:11.202025 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ae66db9-c129-4a17-bd99-9284ae2e40ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:11.202025 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ae66db9-c129-4a17-bd99-9284ae2e40ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:11.202025 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ae66db9-c129-4a17-bd99-9284ae2e40ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:11.242203 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7ae66db9-c129-4a17-bd99-9284ae2e40ae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062801 Aug 07 01:23:11.242683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 725/1383] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:23:11 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:11.615782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e0b94482-6efd-459e-a541-5b4baa658c5b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Action: 'create', calling method: .version_select at 0x7f1fad6995e0>, body: {"keypair": {"name": "tempest-keypair-928595154", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE77YNA2RYUP7wts/EO00bhepfh4v054xAfYNU8EEcqn79ThiXic01l1ULKdxIEov6SvY88xrjCo90tsbLwTzdYY/5nEONf2ZbdNLzjiEdTTIP0gEqCNA8DHWSkksyG9ow=="}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:23:11.646954 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-e0b94482-6efd-459e-a541-5b4baa658c5b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for project 35820b4f79774ac49f43fce915f56d6f. Resources: {'key_pairs'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:23:11.656378 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-e0b94482-6efd-459e-a541-5b4baa658c5b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for user e65f0f1ba54f483abff788974b1256cb and project 35820b4f79774ac49f43fce915f56d6f. Resources: {'key_pairs'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:23:11.701559 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e0b94482-6efd-459e-a541-5b4baa658c5b tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.093942 Aug 07 01:23:11.702848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 726/1384] 10.222.0.26 () {60 vars in 1159 bytes} [Sun Aug 7 01:23:11 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:12.218910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-54758dc7-b811-4562-ace8-73165314e63b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:12.220089 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-54758dc7-b811-4562-ace8-73165314e63b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:12.231781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54758dc7-b811-4562-ace8-73165314e63b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:12.232843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54758dc7-b811-4562-ace8-73165314e63b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:12.233752 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54758dc7-b811-4562-ace8-73165314e63b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:12.235075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54758dc7-b811-4562-ace8-73165314e63b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:12.235956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54758dc7-b811-4562-ace8-73165314e63b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:12.236855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54758dc7-b811-4562-ace8-73165314e63b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:12.350127 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-54758dc7-b811-4562-ace8-73165314e63b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:12.360472 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-54758dc7-b811-4562-ace8-73165314e63b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.150427 Aug 07 01:23:12.360978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 727/1385] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:12 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:12.366199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-25de6aa0-2b33-4032-a60b-cee4c073472d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:12.372656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-25de6aa0-2b33-4032-a60b-cee4c073472d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:12.372656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=2,nova_api:DELETE=3,nova_api:INSERT=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:12.377804 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25de6aa0-2b33-4032-a60b-cee4c073472d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:12.378483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25de6aa0-2b33-4032-a60b-cee4c073472d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:12.378995 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25de6aa0-2b33-4032-a60b-cee4c073472d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:12.379843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25de6aa0-2b33-4032-a60b-cee4c073472d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:12.380472 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25de6aa0-2b33-4032-a60b-cee4c073472d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:12.381023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25de6aa0-2b33-4032-a60b-cee4c073472d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:12.424676 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-25de6aa0-2b33-4032-a60b-cee4c073472d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062466 Aug 07 01:23:12.425810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 728/1386] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:23:12 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:13.400601 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9f7167f3-84b8-4767-8591-1b964c2a02cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:13.401623 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9f7167f3-84b8-4767-8591-1b964c2a02cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:13.410417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9f7167f3-84b8-4767-8591-1b964c2a02cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:13.410971 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9f7167f3-84b8-4767-8591-1b964c2a02cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:13.411460 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9f7167f3-84b8-4767-8591-1b964c2a02cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:13.412303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9f7167f3-84b8-4767-8591-1b964c2a02cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:13.412768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9f7167f3-84b8-4767-8591-1b964c2a02cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:13.413685 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9f7167f3-84b8-4767-8591-1b964c2a02cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:13.558773 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9f7167f3-84b8-4767-8591-1b964c2a02cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:13.566993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9f7167f3-84b8-4767-8591-1b964c2a02cf tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.171189 Aug 07 01:23:13.567873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 729/1387] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:13 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:13.584285 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b5ece142-6547-4675-b427-0bfed08b15cf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:13.584285 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b5ece142-6547-4675-b427-0bfed08b15cf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:13.592425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5ece142-6547-4675-b427-0bfed08b15cf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:13.593894 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5ece142-6547-4675-b427-0bfed08b15cf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:13.594664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5ece142-6547-4675-b427-0bfed08b15cf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:13.595962 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5ece142-6547-4675-b427-0bfed08b15cf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:13.596828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5ece142-6547-4675-b427-0bfed08b15cf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:13.597618 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5ece142-6547-4675-b427-0bfed08b15cf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:13.663919 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b5ece142-6547-4675-b427-0bfed08b15cf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.093940 Aug 07 01:23:13.665454 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 730/1388] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:23:13 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:14.062888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-26838487", "imageRef": "a5f360f4-1500-4970-aefe-b08a6645fb80", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "04909372-4f17-4c44-b384-6e7c31434dac"}], "security_groups": [{"name": "tempest-securitygroup--1195454200"}], "key_name": "tempest-keypair-928595154", "user_data": "}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:23:14.184534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.network.neutron [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] validate_networks() for [('04909372-4f17-4c44-b384-6e7c31434dac', None, None, None, None, None)] {{(pid=117983) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:23:14.462855 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=62,nova_api:UPDATE=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:14.491895 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=38,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:14.765093 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Flavor limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:23:14.765791 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Image limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:23:14.766583 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Flavor pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:23:14.767236 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Image pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:23:14.767830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:23:14.783722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:14.784477 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:14.785157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:14.786473 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:14.787181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:14.787861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:14.819804 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for project 35820b4f79774ac49f43fce915f56d6f. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:23:14.828266 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Getting quotas for user e65f0f1ba54f483abff788974b1256cb and project 35820b4f79774ac49f43fce915f56d6f. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:23:14.849628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Going to run 1 instances... {{(pid=117983) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:23:14.853962 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a5f360f4-1500-4970-aefe-b08a6645fb80',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117983) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:23:14.932350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-76bffe2a-1a6b-4511-96ac-2112fd2ed656 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.875417 Aug 07 01:23:14.933200 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 731/1389] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:23:14 2022] POST /compute/v2.1/servers => generated 402 bytes in 877 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:23:14.939565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a98ea7e6-1d65-48ff-8911-304714135359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:14.941153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a98ea7e6-1d65-48ff-8911-304714135359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:14.948198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a98ea7e6-1d65-48ff-8911-304714135359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:14.948729 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a98ea7e6-1d65-48ff-8911-304714135359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:14.949438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a98ea7e6-1d65-48ff-8911-304714135359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:14.950453 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a98ea7e6-1d65-48ff-8911-304714135359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:14.950977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a98ea7e6-1d65-48ff-8911-304714135359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:14.951488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a98ea7e6-1d65-48ff-8911-304714135359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:15.046252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a98ea7e6-1d65-48ff-8911-304714135359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:15.055512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a98ea7e6-1d65-48ff-8911-304714135359 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.121387 Aug 07 01:23:15.056144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 732/1390] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:14 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:15.062181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4e3b402f-b35f-407d-a714-d8f1522855a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:15.064704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4e3b402f-b35f-407d-a714-d8f1522855a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:15.073355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e3b402f-b35f-407d-a714-d8f1522855a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:15.073775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e3b402f-b35f-407d-a714-d8f1522855a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:15.074498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e3b402f-b35f-407d-a714-d8f1522855a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:15.075367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e3b402f-b35f-407d-a714-d8f1522855a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:15.076058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e3b402f-b35f-407d-a714-d8f1522855a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:15.077242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e3b402f-b35f-407d-a714-d8f1522855a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:15.112621 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4e3b402f-b35f-407d-a714-d8f1522855a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055025 Aug 07 01:23:15.113213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 733/1391] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:23:15 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:15.118017 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a92cdb88-bd10-425d-b927-7d7fca853cda tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:15.118630 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a92cdb88-bd10-425d-b927-7d7fca853cda tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:15.198668 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a92cdb88-bd10-425d-b927-7d7fca853cda tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:15.209041 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a92cdb88-bd10-425d-b927-7d7fca853cda tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 200 len: 1239 microversion: 2.1 time: 0.094560 Aug 07 01:23:15.210012 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 734/1392] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:15 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 1239 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:16.091614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e63c6027-21ad-45c0-b114-301d857d7936 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:16.093268 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e63c6027-21ad-45c0-b114-301d857d7936 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:16.099814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e63c6027-21ad-45c0-b114-301d857d7936 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:16.100342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e63c6027-21ad-45c0-b114-301d857d7936 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:16.100796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e63c6027-21ad-45c0-b114-301d857d7936 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:16.101517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e63c6027-21ad-45c0-b114-301d857d7936 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:16.101976 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e63c6027-21ad-45c0-b114-301d857d7936 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:16.102407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e63c6027-21ad-45c0-b114-301d857d7936 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:16.177629 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e63c6027-21ad-45c0-b114-301d857d7936 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:16.191807 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e63c6027-21ad-45c0-b114-301d857d7936 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.104065 Aug 07 01:23:16.193011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 735/1393] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:16 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:16.202061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1a43d5d5-06b1-4048-a0a6-e1a683b144ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:23:16.205913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1a43d5d5-06b1-4048-a0a6-e1a683b144ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:16.217084 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a43d5d5-06b1-4048-a0a6-e1a683b144ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:16.218425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a43d5d5-06b1-4048-a0a6-e1a683b144ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:16.219177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a43d5d5-06b1-4048-a0a6-e1a683b144ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:16.220784 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a43d5d5-06b1-4048-a0a6-e1a683b144ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:16.221605 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a43d5d5-06b1-4048-a0a6-e1a683b144ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:16.222406 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a43d5d5-06b1-4048-a0a6-e1a683b144ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:16.230487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:16.329174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1a43d5d5-06b1-4048-a0a6-e1a683b144ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "POST /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/action" status: 200 len: 74389 microversion: 2.1 time: 0.133806 Aug 07 01:23:16.329537 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 736/1394] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:23:16 2022] POST /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd/action => generated 74389 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 07 01:23:16.335511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6633ad27-4dc2-4e0d-9bd5-f047a8af3269 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:16.336426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6633ad27-4dc2-4e0d-9bd5-f047a8af3269 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:16.348563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6633ad27-4dc2-4e0d-9bd5-f047a8af3269 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:16.349181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6633ad27-4dc2-4e0d-9bd5-f047a8af3269 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:16.349746 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6633ad27-4dc2-4e0d-9bd5-f047a8af3269 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:16.351123 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6633ad27-4dc2-4e0d-9bd5-f047a8af3269 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:16.351722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6633ad27-4dc2-4e0d-9bd5-f047a8af3269 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:16.352292 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6633ad27-4dc2-4e0d-9bd5-f047a8af3269 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:16.475390 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6633ad27-4dc2-4e0d-9bd5-f047a8af3269 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:16.488535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6633ad27-4dc2-4e0d-9bd5-f047a8af3269 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 200 len: 1309 microversion: 2.1 time: 0.157424 Aug 07 01:23:16.489536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 737/1395] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:16 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 1309 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:16.957478 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.objects.instance [None req-c7be3fee-cb75-40d1-9094-90ccc1c38d47 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lazy-loading 'services' on Instance uuid f481f4cc-2803-4f12-9cd2-db6055d1488c {{(pid=117984) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:23:17.221549 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ef7c4a30-f687-4cb3-b9f6-29a0851bb278 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:17.222536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ef7c4a30-f687-4cb3-b9f6-29a0851bb278 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:17.231036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef7c4a30-f687-4cb3-b9f6-29a0851bb278 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:17.231701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef7c4a30-f687-4cb3-b9f6-29a0851bb278 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:17.232375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef7c4a30-f687-4cb3-b9f6-29a0851bb278 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:17.233213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef7c4a30-f687-4cb3-b9f6-29a0851bb278 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:17.234417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef7c4a30-f687-4cb3-b9f6-29a0851bb278 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:17.234957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef7c4a30-f687-4cb3-b9f6-29a0851bb278 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:17.331576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ef7c4a30-f687-4cb3-b9f6-29a0851bb278 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:17.348292 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ef7c4a30-f687-4cb3-b9f6-29a0851bb278 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.131645 Aug 07 01:23:17.358961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 738/1396] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:17 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:17.514263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c996c6f8-ec52-48e1-afee-f9314e795519 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:17.515602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c996c6f8-ec52-48e1-afee-f9314e795519 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:17.524897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c996c6f8-ec52-48e1-afee-f9314e795519 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:17.525631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c996c6f8-ec52-48e1-afee-f9314e795519 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:17.526299 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c996c6f8-ec52-48e1-afee-f9314e795519 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:17.527327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c996c6f8-ec52-48e1-afee-f9314e795519 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:17.528018 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c996c6f8-ec52-48e1-afee-f9314e795519 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:17.528693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c996c6f8-ec52-48e1-afee-f9314e795519 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:17.753460 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c996c6f8-ec52-48e1-afee-f9314e795519 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:17.763489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c996c6f8-ec52-48e1-afee-f9314e795519 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 200 len: 1365 microversion: 2.1 time: 0.254059 Aug 07 01:23:17.764304 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 739/1397] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:17 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 1365 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:18.382864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-13e8a06f-178a-42d4-9905-ae8b4e7e3bcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:18.384357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-13e8a06f-178a-42d4-9905-ae8b4e7e3bcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:18.396757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-13e8a06f-178a-42d4-9905-ae8b4e7e3bcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:18.397605 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-13e8a06f-178a-42d4-9905-ae8b4e7e3bcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:18.398391 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-13e8a06f-178a-42d4-9905-ae8b4e7e3bcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:18.399714 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-13e8a06f-178a-42d4-9905-ae8b4e7e3bcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:18.400827 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-13e8a06f-178a-42d4-9905-ae8b4e7e3bcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:18.401864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-13e8a06f-178a-42d4-9905-ae8b4e7e3bcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:18.517811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-13e8a06f-178a-42d4-9905-ae8b4e7e3bcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:18.528584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-13e8a06f-178a-42d4-9905-ae8b4e7e3bcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.152052 Aug 07 01:23:18.529384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 740/1398] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:18 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:18.536009 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-fad203cd-c0dc-4021-b0f1-16da19eebd78 req-ed25b98b-98f2-4739-b22b-2af0e0905472 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1143cfd8-c86b-4023-84cc-8518db39a1be", "tag": "65d03541-9d7a-43bd-a6b1-fd6ae8b03113"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:23:18.548227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-fad203cd-c0dc-4021-b0f1-16da19eebd78 req-ed25b98b-98f2-4739-b22b-2af0e0905472 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:18.548783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-fad203cd-c0dc-4021-b0f1-16da19eebd78 req-ed25b98b-98f2-4739-b22b-2af0e0905472 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:18.549366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-fad203cd-c0dc-4021-b0f1-16da19eebd78 req-ed25b98b-98f2-4739-b22b-2af0e0905472 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:18.562715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-fad203cd-c0dc-4021-b0f1-16da19eebd78 req-ed25b98b-98f2-4739-b22b-2af0e0905472 service nova] Creating event network-changed:65d03541-9d7a-43bd-a6b1-fd6ae8b03113 for instance 1143cfd8-c86b-4023-84cc-8518db39a1be on hv-cinder-maste Aug 07 01:23:18.568130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-fad203cd-c0dc-4021-b0f1-16da19eebd78 req-ed25b98b-98f2-4739-b22b-2af0e0905472 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037080 Aug 07 01:23:18.569014 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 741/1399] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:23:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:18.788963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f8706409-bec2-4c45-9133-74ef9099d293 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:18.789673 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f8706409-bec2-4c45-9133-74ef9099d293 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:18.797477 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8706409-bec2-4c45-9133-74ef9099d293 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:18.797978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8706409-bec2-4c45-9133-74ef9099d293 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:18.798418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8706409-bec2-4c45-9133-74ef9099d293 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:18.799144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8706409-bec2-4c45-9133-74ef9099d293 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:18.799596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8706409-bec2-4c45-9133-74ef9099d293 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:18.800020 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8706409-bec2-4c45-9133-74ef9099d293 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:18.974690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f8706409-bec2-4c45-9133-74ef9099d293 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:18.984566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f8706409-bec2-4c45-9133-74ef9099d293 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 200 len: 1365 microversion: 2.1 time: 0.200843 Aug 07 01:23:18.985170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 742/1400] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:18 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 1365 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:19.555724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8b039aeb-ed17-441e-8f6c-e6d2d9b9b167 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:19.556438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8b039aeb-ed17-441e-8f6c-e6d2d9b9b167 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:19.563675 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b039aeb-ed17-441e-8f6c-e6d2d9b9b167 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:19.564170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b039aeb-ed17-441e-8f6c-e6d2d9b9b167 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:19.564644 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b039aeb-ed17-441e-8f6c-e6d2d9b9b167 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:19.565371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b039aeb-ed17-441e-8f6c-e6d2d9b9b167 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:19.565823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b039aeb-ed17-441e-8f6c-e6d2d9b9b167 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:19.566250 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b039aeb-ed17-441e-8f6c-e6d2d9b9b167 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:19.639154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8b039aeb-ed17-441e-8f6c-e6d2d9b9b167 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:19.648809 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8b039aeb-ed17-441e-8f6c-e6d2d9b9b167 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097047 Aug 07 01:23:19.649362 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 743/1401] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:19 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:20.007812 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-edfc0f04-dec4-47da-bca5-48821149efa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:20.009256 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-edfc0f04-dec4-47da-bca5-48821149efa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:20.015822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-edfc0f04-dec4-47da-bca5-48821149efa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:20.016431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-edfc0f04-dec4-47da-bca5-48821149efa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:20.016985 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-edfc0f04-dec4-47da-bca5-48821149efa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:20.017821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-edfc0f04-dec4-47da-bca5-48821149efa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:20.018383 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-edfc0f04-dec4-47da-bca5-48821149efa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:20.019050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-edfc0f04-dec4-47da-bca5-48821149efa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:20.203156 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-edfc0f04-dec4-47da-bca5-48821149efa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:20.211809 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-edfc0f04-dec4-47da-bca5-48821149efa2 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 200 len: 1365 microversion: 2.1 time: 0.208482 Aug 07 01:23:20.212524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 744/1402] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:20 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 1365 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:20.675199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5e982aa7-185d-47ad-be9e-77953a2ac339 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:20.676521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5e982aa7-185d-47ad-be9e-77953a2ac339 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:20.687832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5e982aa7-185d-47ad-be9e-77953a2ac339 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:20.688624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5e982aa7-185d-47ad-be9e-77953a2ac339 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:20.689329 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5e982aa7-185d-47ad-be9e-77953a2ac339 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:20.690154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5e982aa7-185d-47ad-be9e-77953a2ac339 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:20.690628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5e982aa7-185d-47ad-be9e-77953a2ac339 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:20.691060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5e982aa7-185d-47ad-be9e-77953a2ac339 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:20.775817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5e982aa7-185d-47ad-be9e-77953a2ac339 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:20.786113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5e982aa7-185d-47ad-be9e-77953a2ac339 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.118582 Aug 07 01:23:20.786726 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 745/1403] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:20 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:21.248670 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6cfe47b3-60e1-4f5c-b516-d43b13f6f105 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:21.249396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6cfe47b3-60e1-4f5c-b516-d43b13f6f105 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:21.256561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6cfe47b3-60e1-4f5c-b516-d43b13f6f105 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:21.257081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6cfe47b3-60e1-4f5c-b516-d43b13f6f105 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:21.257522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6cfe47b3-60e1-4f5c-b516-d43b13f6f105 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:21.258242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6cfe47b3-60e1-4f5c-b516-d43b13f6f105 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:21.258697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6cfe47b3-60e1-4f5c-b516-d43b13f6f105 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:21.259346 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6cfe47b3-60e1-4f5c-b516-d43b13f6f105 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:21.424795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6cfe47b3-60e1-4f5c-b516-d43b13f6f105 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:21.433240 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6cfe47b3-60e1-4f5c-b516-d43b13f6f105 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 200 len: 1365 microversion: 2.1 time: 0.188713 Aug 07 01:23:21.433840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 746/1404] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:21 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 1365 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:21.812459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0e74366b-30f4-4281-9279-2c432eeda0f7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:21.813607 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0e74366b-30f4-4281-9279-2c432eeda0f7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:21.825804 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e74366b-30f4-4281-9279-2c432eeda0f7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:21.826668 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e74366b-30f4-4281-9279-2c432eeda0f7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:21.828505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e74366b-30f4-4281-9279-2c432eeda0f7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:21.829738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e74366b-30f4-4281-9279-2c432eeda0f7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:21.830560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e74366b-30f4-4281-9279-2c432eeda0f7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:21.831320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e74366b-30f4-4281-9279-2c432eeda0f7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:21.941489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0e74366b-30f4-4281-9279-2c432eeda0f7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:21.958297 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0e74366b-30f4-4281-9279-2c432eeda0f7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.153215 Aug 07 01:23:21.959399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 747/1405] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:21 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:22.456792 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8b8ba4c9-bd14-4767-8c97-778065497fe9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:22.457440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8b8ba4c9-bd14-4767-8c97-778065497fe9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:22.465644 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b8ba4c9-bd14-4767-8c97-778065497fe9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:22.466201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b8ba4c9-bd14-4767-8c97-778065497fe9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:22.466860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b8ba4c9-bd14-4767-8c97-778065497fe9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:22.467881 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b8ba4c9-bd14-4767-8c97-778065497fe9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:22.468488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b8ba4c9-bd14-4767-8c97-778065497fe9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:22.468945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8b8ba4c9-bd14-4767-8c97-778065497fe9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:22.665181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8b8ba4c9-bd14-4767-8c97-778065497fe9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:22.680404 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8b8ba4c9-bd14-4767-8c97-778065497fe9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 200 len: 1365 microversion: 2.1 time: 0.228492 Aug 07 01:23:22.681101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 748/1406] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:22 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 1365 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:22.991724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b9f1dd55-d196-4466-96c1-fa6dcb8f6855 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:22.992805 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b9f1dd55-d196-4466-96c1-fa6dcb8f6855 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:23.004080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9f1dd55-d196-4466-96c1-fa6dcb8f6855 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:23.005026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9f1dd55-d196-4466-96c1-fa6dcb8f6855 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:23.005847 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9f1dd55-d196-4466-96c1-fa6dcb8f6855 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:23.006873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9f1dd55-d196-4466-96c1-fa6dcb8f6855 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:23.007389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9f1dd55-d196-4466-96c1-fa6dcb8f6855 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:23.007826 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9f1dd55-d196-4466-96c1-fa6dcb8f6855 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:23.084758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b9f1dd55-d196-4466-96c1-fa6dcb8f6855 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:23.093052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b9f1dd55-d196-4466-96c1-fa6dcb8f6855 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.105882 Aug 07 01:23:23.093652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 749/1407] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:22 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:23.704156 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fcde1a72-4214-4921-ae90-827e28c07926 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:23.704851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fcde1a72-4214-4921-ae90-827e28c07926 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:23.711403 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcde1a72-4214-4921-ae90-827e28c07926 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:23.711899 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcde1a72-4214-4921-ae90-827e28c07926 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:23.712377 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcde1a72-4214-4921-ae90-827e28c07926 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:23.713113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcde1a72-4214-4921-ae90-827e28c07926 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:23.713642 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcde1a72-4214-4921-ae90-827e28c07926 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:23.714056 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcde1a72-4214-4921-ae90-827e28c07926 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:23.879700 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-fcde1a72-4214-4921-ae90-827e28c07926 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:23.901025 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fcde1a72-4214-4921-ae90-827e28c07926 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 200 len: 1365 microversion: 2.1 time: 0.200381 Aug 07 01:23:23.902337 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 750/1408] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:23 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 1365 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:24.115818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-aad70b1e-0968-4890-b242-0255460c7288 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:24.116913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-aad70b1e-0968-4890-b242-0255460c7288 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:24.124989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aad70b1e-0968-4890-b242-0255460c7288 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:24.125828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aad70b1e-0968-4890-b242-0255460c7288 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:24.126617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aad70b1e-0968-4890-b242-0255460c7288 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:24.128152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aad70b1e-0968-4890-b242-0255460c7288 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:24.129521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aad70b1e-0968-4890-b242-0255460c7288 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:24.129658 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aad70b1e-0968-4890-b242-0255460c7288 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:24.251936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-aad70b1e-0968-4890-b242-0255460c7288 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:24.261747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-aad70b1e-0968-4890-b242-0255460c7288 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.150234 Aug 07 01:23:24.262604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 751/1409] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:24 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:24.370996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-359265b8-3af8-423a-b1e2-bc004b4d5796 req-0e0dd7b4-a540-414d-8670-79380e4d0e92 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1143cfd8-c86b-4023-84cc-8518db39a1be", "name": "network-vif-plugged", "status": "completed", "tag": "65d03541-9d7a-43bd-a6b1-fd6ae8b03113"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:23:24.383213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-359265b8-3af8-423a-b1e2-bc004b4d5796 req-0e0dd7b4-a540-414d-8670-79380e4d0e92 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:24.383708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-359265b8-3af8-423a-b1e2-bc004b4d5796 req-0e0dd7b4-a540-414d-8670-79380e4d0e92 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:24.384161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-359265b8-3af8-423a-b1e2-bc004b4d5796 req-0e0dd7b4-a540-414d-8670-79380e4d0e92 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:24.399329 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-359265b8-3af8-423a-b1e2-bc004b4d5796 req-0e0dd7b4-a540-414d-8670-79380e4d0e92 service nova] Creating event network-vif-plugged:65d03541-9d7a-43bd-a6b1-fd6ae8b03113 for instance 1143cfd8-c86b-4023-84cc-8518db39a1be on hv-cinder-maste Aug 07 01:23:24.408723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-359265b8-3af8-423a-b1e2-bc004b4d5796 req-0e0dd7b4-a540-414d-8670-79380e4d0e92 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.144415 Aug 07 01:23:24.409374 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 752/1410] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:23:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:24.923722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:24.929034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-92c8cc3d-b0ac-4351-89b9-e9fa57649d13 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:24.929842 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-92c8cc3d-b0ac-4351-89b9-e9fa57649d13 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:24.938317 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-92c8cc3d-b0ac-4351-89b9-e9fa57649d13 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:24.938868 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-92c8cc3d-b0ac-4351-89b9-e9fa57649d13 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:24.939353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-92c8cc3d-b0ac-4351-89b9-e9fa57649d13 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:24.940131 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-92c8cc3d-b0ac-4351-89b9-e9fa57649d13 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:24.940692 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-92c8cc3d-b0ac-4351-89b9-e9fa57649d13 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:24.941168 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-92c8cc3d-b0ac-4351-89b9-e9fa57649d13 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:25.108819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-92c8cc3d-b0ac-4351-89b9-e9fa57649d13 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:25.117227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-92c8cc3d-b0ac-4351-89b9-e9fa57649d13 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 200 len: 1365 microversion: 2.1 time: 0.194630 Aug 07 01:23:25.117970 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 753/1411] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:24 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 1365 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:25.299262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f08ade25-4ced-4a00-af47-597d15a348e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:25.299860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f08ade25-4ced-4a00-af47-597d15a348e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:25.302415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=12 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:25.308831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f08ade25-4ced-4a00-af47-597d15a348e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:25.309198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f08ade25-4ced-4a00-af47-597d15a348e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:25.309713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f08ade25-4ced-4a00-af47-597d15a348e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:25.310485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f08ade25-4ced-4a00-af47-597d15a348e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:25.310974 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f08ade25-4ced-4a00-af47-597d15a348e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:25.312120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f08ade25-4ced-4a00-af47-597d15a348e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:25.336742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:25.397034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f08ade25-4ced-4a00-af47-597d15a348e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:25.406541 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f08ade25-4ced-4a00-af47-597d15a348e7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.111814 Aug 07 01:23:25.407153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 754/1412] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:25 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:26.140752 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3f600ddb-44a8-4403-912c-5e0127f8a21c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:26.141640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3f600ddb-44a8-4403-912c-5e0127f8a21c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:26.149179 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f600ddb-44a8-4403-912c-5e0127f8a21c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:26.149747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f600ddb-44a8-4403-912c-5e0127f8a21c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:26.150226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f600ddb-44a8-4403-912c-5e0127f8a21c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:26.151060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f600ddb-44a8-4403-912c-5e0127f8a21c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:26.151684 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f600ddb-44a8-4403-912c-5e0127f8a21c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:26.152213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f600ddb-44a8-4403-912c-5e0127f8a21c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:26.319342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3f600ddb-44a8-4403-912c-5e0127f8a21c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:26.329787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3f600ddb-44a8-4403-912c-5e0127f8a21c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 200 len: 1365 microversion: 2.1 time: 0.193582 Aug 07 01:23:26.330713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 755/1413] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:26 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 1365 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:26.431905 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d4ebd69f-3117-4c37-a303-ead7e3309f93 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:26.432326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d4ebd69f-3117-4c37-a303-ead7e3309f93 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:26.440793 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4ebd69f-3117-4c37-a303-ead7e3309f93 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:26.441436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4ebd69f-3117-4c37-a303-ead7e3309f93 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:26.441845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4ebd69f-3117-4c37-a303-ead7e3309f93 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:26.442674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4ebd69f-3117-4c37-a303-ead7e3309f93 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:26.443318 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4ebd69f-3117-4c37-a303-ead7e3309f93 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:26.443860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4ebd69f-3117-4c37-a303-ead7e3309f93 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:26.518417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d4ebd69f-3117-4c37-a303-ead7e3309f93 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:26.526997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d4ebd69f-3117-4c37-a303-ead7e3309f93 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100526 Aug 07 01:23:26.527604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 756/1414] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:26 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:26.963388 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=4 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:27.355506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-81995370-c389-4e7a-b5e9-66b7a8b433db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:27.356303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-81995370-c389-4e7a-b5e9-66b7a8b433db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:27.364632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81995370-c389-4e7a-b5e9-66b7a8b433db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:27.365534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81995370-c389-4e7a-b5e9-66b7a8b433db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:27.366290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81995370-c389-4e7a-b5e9-66b7a8b433db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:27.367078 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81995370-c389-4e7a-b5e9-66b7a8b433db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:27.367649 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81995370-c389-4e7a-b5e9-66b7a8b433db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:27.368188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81995370-c389-4e7a-b5e9-66b7a8b433db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:27.539420 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-81995370-c389-4e7a-b5e9-66b7a8b433db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:27.549501 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-81995370-c389-4e7a-b5e9-66b7a8b433db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 200 len: 1555 microversion: 2.1 time: 0.198510 Aug 07 01:23:27.550170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 757/1415] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:27 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 1555 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:27.555710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fda1bf5d-e652-494b-8024-7ed0bf9fa317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:27.556592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fda1bf5d-e652-494b-8024-7ed0bf9fa317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:27.564709 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fda1bf5d-e652-494b-8024-7ed0bf9fa317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:27.565380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fda1bf5d-e652-494b-8024-7ed0bf9fa317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:27.566106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fda1bf5d-e652-494b-8024-7ed0bf9fa317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:27.566967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fda1bf5d-e652-494b-8024-7ed0bf9fa317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:27.567740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fda1bf5d-e652-494b-8024-7ed0bf9fa317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:27.568340 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fda1bf5d-e652-494b-8024-7ed0bf9fa317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:27.642510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-fda1bf5d-e652-494b-8024-7ed0bf9fa317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:27.651524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fda1bf5d-e652-494b-8024-7ed0bf9fa317 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100130 Aug 07 01:23:27.652119 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 758/1416] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:27 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:27.658074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0cd2b977-536a-48ec-b166-582f460dee0c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:27.658576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0cd2b977-536a-48ec-b166-582f460dee0c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:27.668023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cd2b977-536a-48ec-b166-582f460dee0c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:27.668956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cd2b977-536a-48ec-b166-582f460dee0c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:27.669722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cd2b977-536a-48ec-b166-582f460dee0c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:27.671732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cd2b977-536a-48ec-b166-582f460dee0c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:27.672638 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cd2b977-536a-48ec-b166-582f460dee0c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:27.673464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cd2b977-536a-48ec-b166-582f460dee0c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:27.831571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0cd2b977-536a-48ec-b166-582f460dee0c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.178290 Aug 07 01:23:27.832166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 759/1417] 10.222.0.26 () {58 vars in 1275 bytes} [Sun Aug 7 01:23:27 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be/os-interface => generated 284 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:23:27.994568 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:23:27.995283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:23:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518 (ip 10.222.0.26) !!! Aug 07 01:23:28.674835 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e6a7b753-233c-41b5-b2cc-e476b6009594 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:28.675504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e6a7b753-233c-41b5-b2cc-e476b6009594 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:28.683710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6a7b753-233c-41b5-b2cc-e476b6009594 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:28.684251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6a7b753-233c-41b5-b2cc-e476b6009594 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:28.684752 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6a7b753-233c-41b5-b2cc-e476b6009594 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:28.685544 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6a7b753-233c-41b5-b2cc-e476b6009594 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:28.686237 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6a7b753-233c-41b5-b2cc-e476b6009594 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:28.686701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6a7b753-233c-41b5-b2cc-e476b6009594 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:28.760502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e6a7b753-233c-41b5-b2cc-e476b6009594 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:28.770183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e6a7b753-233c-41b5-b2cc-e476b6009594 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100525 Aug 07 01:23:28.770790 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 760/1418] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:28 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:29.425075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-718e273c-8f8f-4f7b-9558-269a15b3b521 req-09e2b067-1d53-47dd-a8f8-3efdce3f8c49 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1143cfd8-c86b-4023-84cc-8518db39a1be", "tag": "65d03541-9d7a-43bd-a6b1-fd6ae8b03113"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:23:29.434977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-718e273c-8f8f-4f7b-9558-269a15b3b521 req-09e2b067-1d53-47dd-a8f8-3efdce3f8c49 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:29.435602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-718e273c-8f8f-4f7b-9558-269a15b3b521 req-09e2b067-1d53-47dd-a8f8-3efdce3f8c49 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:29.436851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-718e273c-8f8f-4f7b-9558-269a15b3b521 req-09e2b067-1d53-47dd-a8f8-3efdce3f8c49 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:29.451490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-718e273c-8f8f-4f7b-9558-269a15b3b521 req-09e2b067-1d53-47dd-a8f8-3efdce3f8c49 service nova] Creating event network-changed:65d03541-9d7a-43bd-a6b1-fd6ae8b03113 for instance 1143cfd8-c86b-4023-84cc-8518db39a1be on hv-cinder-maste Aug 07 01:23:29.458363 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-718e273c-8f8f-4f7b-9558-269a15b3b521 req-09e2b067-1d53-47dd-a8f8-3efdce3f8c49 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036699 Aug 07 01:23:29.459370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 761/1419] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:23:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:23:29.795982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c44642a9-de61-4570-85b9-40a41f67acd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:29.797205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c44642a9-de61-4570-85b9-40a41f67acd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:29.807911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c44642a9-de61-4570-85b9-40a41f67acd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:29.808887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c44642a9-de61-4570-85b9-40a41f67acd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:29.809653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c44642a9-de61-4570-85b9-40a41f67acd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:29.811521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c44642a9-de61-4570-85b9-40a41f67acd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:29.812443 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c44642a9-de61-4570-85b9-40a41f67acd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:29.813221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c44642a9-de61-4570-85b9-40a41f67acd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:29.921411 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c44642a9-de61-4570-85b9-40a41f67acd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:29.934772 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c44642a9-de61-4570-85b9-40a41f67acd6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.144785 Aug 07 01:23:29.935763 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 762/1420] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:29 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:30.970427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a40510a0-1345-41c8-b5de-ad070e29e7f0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:30.971754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a40510a0-1345-41c8-b5de-ad070e29e7f0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:30.984361 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a40510a0-1345-41c8-b5de-ad070e29e7f0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:30.985523 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a40510a0-1345-41c8-b5de-ad070e29e7f0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:30.986420 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a40510a0-1345-41c8-b5de-ad070e29e7f0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:30.987828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a40510a0-1345-41c8-b5de-ad070e29e7f0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:30.988827 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a40510a0-1345-41c8-b5de-ad070e29e7f0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:30.989689 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a40510a0-1345-41c8-b5de-ad070e29e7f0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:31.106839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a40510a0-1345-41c8-b5de-ad070e29e7f0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:31.118318 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a40510a0-1345-41c8-b5de-ad070e29e7f0 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.155475 Aug 07 01:23:31.118982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 763/1421] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:30 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:32.140893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-18c9ebf1-d83f-4dcd-a027-924e0fc86cbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:32.141706 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-18c9ebf1-d83f-4dcd-a027-924e0fc86cbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:32.149106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-18c9ebf1-d83f-4dcd-a027-924e0fc86cbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:32.149606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-18c9ebf1-d83f-4dcd-a027-924e0fc86cbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:32.150044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-18c9ebf1-d83f-4dcd-a027-924e0fc86cbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:32.150773 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-18c9ebf1-d83f-4dcd-a027-924e0fc86cbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:32.151274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-18c9ebf1-d83f-4dcd-a027-924e0fc86cbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:32.151707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-18c9ebf1-d83f-4dcd-a027-924e0fc86cbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:32.221521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-18c9ebf1-d83f-4dcd-a027-924e0fc86cbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:32.230511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-18c9ebf1-d83f-4dcd-a027-924e0fc86cbc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.094071 Aug 07 01:23:32.231128 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 764/1422] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:32 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:33.266235 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d828783e-9de0-4f5c-9496-245116108fb8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:33.266991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d828783e-9de0-4f5c-9496-245116108fb8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:33.275004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d828783e-9de0-4f5c-9496-245116108fb8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:33.276500 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d828783e-9de0-4f5c-9496-245116108fb8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:33.276500 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d828783e-9de0-4f5c-9496-245116108fb8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:33.277131 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d828783e-9de0-4f5c-9496-245116108fb8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:33.278259 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d828783e-9de0-4f5c-9496-245116108fb8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:33.278458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d828783e-9de0-4f5c-9496-245116108fb8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:33.351733 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d828783e-9de0-4f5c-9496-245116108fb8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:33.361472 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d828783e-9de0-4f5c-9496-245116108fb8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100090 Aug 07 01:23:33.362068 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 765/1423] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:33 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:34.383718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9cf1f482-2c91-4a57-8095-e8546a77c295 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:34.384674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9cf1f482-2c91-4a57-8095-e8546a77c295 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:34.392770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9cf1f482-2c91-4a57-8095-e8546a77c295 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:34.393302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9cf1f482-2c91-4a57-8095-e8546a77c295 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:34.393751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9cf1f482-2c91-4a57-8095-e8546a77c295 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:34.394955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9cf1f482-2c91-4a57-8095-e8546a77c295 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:34.395463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9cf1f482-2c91-4a57-8095-e8546a77c295 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:34.395866 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9cf1f482-2c91-4a57-8095-e8546a77c295 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:34.468312 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9cf1f482-2c91-4a57-8095-e8546a77c295 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:34.477349 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9cf1f482-2c91-4a57-8095-e8546a77c295 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097996 Aug 07 01:23:34.477956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 766/1424] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:34 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:35.501738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2512f1f4-e1f5-4516-8622-e73b2eb40159 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:35.502551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2512f1f4-e1f5-4516-8622-e73b2eb40159 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:35.509612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2512f1f4-e1f5-4516-8622-e73b2eb40159 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:35.510116 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2512f1f4-e1f5-4516-8622-e73b2eb40159 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:35.510569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2512f1f4-e1f5-4516-8622-e73b2eb40159 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:35.511302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2512f1f4-e1f5-4516-8622-e73b2eb40159 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:35.511761 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2512f1f4-e1f5-4516-8622-e73b2eb40159 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:35.512227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2512f1f4-e1f5-4516-8622-e73b2eb40159 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:35.610374 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2512f1f4-e1f5-4516-8622-e73b2eb40159 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:35.619007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2512f1f4-e1f5-4516-8622-e73b2eb40159 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.121187 Aug 07 01:23:35.619595 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 767/1425] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:35 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:36.641508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-aa1577f9-253c-47bd-982f-9db04f4a3661 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:36.642511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-aa1577f9-253c-47bd-982f-9db04f4a3661 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:36.649919 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa1577f9-253c-47bd-982f-9db04f4a3661 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:36.650520 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa1577f9-253c-47bd-982f-9db04f4a3661 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:36.651075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa1577f9-253c-47bd-982f-9db04f4a3661 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:36.652022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa1577f9-253c-47bd-982f-9db04f4a3661 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:36.652585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa1577f9-253c-47bd-982f-9db04f4a3661 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:36.653074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa1577f9-253c-47bd-982f-9db04f4a3661 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:36.723999 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-aa1577f9-253c-47bd-982f-9db04f4a3661 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:36.732155 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-aa1577f9-253c-47bd-982f-9db04f4a3661 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.095252 Aug 07 01:23:36.733334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 768/1426] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:36 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:37.755242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bd98498a-8d4e-4ad2-886e-3a43d8f6d320 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:37.756013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bd98498a-8d4e-4ad2-886e-3a43d8f6d320 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:37.758515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:37.765242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bd98498a-8d4e-4ad2-886e-3a43d8f6d320 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:37.765747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bd98498a-8d4e-4ad2-886e-3a43d8f6d320 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:37.766192 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bd98498a-8d4e-4ad2-886e-3a43d8f6d320 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:37.766913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bd98498a-8d4e-4ad2-886e-3a43d8f6d320 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:37.767370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bd98498a-8d4e-4ad2-886e-3a43d8f6d320 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:37.767803 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bd98498a-8d4e-4ad2-886e-3a43d8f6d320 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:37.843032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-bd98498a-8d4e-4ad2-886e-3a43d8f6d320 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:37.851418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bd98498a-8d4e-4ad2-886e-3a43d8f6d320 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100363 Aug 07 01:23:37.852205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 769/1427] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:37 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:38.874317 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ed814e71-6e8b-44fe-b950-84a42e6da077 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:38.875021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ed814e71-6e8b-44fe-b950-84a42e6da077 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:38.882116 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed814e71-6e8b-44fe-b950-84a42e6da077 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:38.882612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed814e71-6e8b-44fe-b950-84a42e6da077 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:38.883057 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed814e71-6e8b-44fe-b950-84a42e6da077 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:38.883804 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed814e71-6e8b-44fe-b950-84a42e6da077 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:38.884294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed814e71-6e8b-44fe-b950-84a42e6da077 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:38.884730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ed814e71-6e8b-44fe-b950-84a42e6da077 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:38.963427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ed814e71-6e8b-44fe-b950-84a42e6da077 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:38.973327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ed814e71-6e8b-44fe-b950-84a42e6da077 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.102903 Aug 07 01:23:38.974111 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 770/1428] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:38 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:39.994881 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-91a8d23f-bd29-421a-aa12-6dc95af8753d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:39.995680 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-91a8d23f-bd29-421a-aa12-6dc95af8753d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:40.002419 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91a8d23f-bd29-421a-aa12-6dc95af8753d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:40.002922 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91a8d23f-bd29-421a-aa12-6dc95af8753d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:40.003364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91a8d23f-bd29-421a-aa12-6dc95af8753d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:40.004093 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91a8d23f-bd29-421a-aa12-6dc95af8753d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:40.004577 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91a8d23f-bd29-421a-aa12-6dc95af8753d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:40.005011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91a8d23f-bd29-421a-aa12-6dc95af8753d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:40.086772 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-91a8d23f-bd29-421a-aa12-6dc95af8753d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:40.102143 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-91a8d23f-bd29-421a-aa12-6dc95af8753d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.110822 Aug 07 01:23:40.103178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 771/1429] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:39 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:41.126441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8516098e-582f-4d58-bfc6-261e28bc935d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:41.127308 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8516098e-582f-4d58-bfc6-261e28bc935d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:41.134221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8516098e-582f-4d58-bfc6-261e28bc935d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:41.134707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8516098e-582f-4d58-bfc6-261e28bc935d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:41.135157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8516098e-582f-4d58-bfc6-261e28bc935d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:41.135882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8516098e-582f-4d58-bfc6-261e28bc935d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:41.136375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8516098e-582f-4d58-bfc6-261e28bc935d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:41.136810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8516098e-582f-4d58-bfc6-261e28bc935d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:41.141259 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:41.202606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8516098e-582f-4d58-bfc6-261e28bc935d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:41.210911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8516098e-582f-4d58-bfc6-261e28bc935d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.089488 Aug 07 01:23:41.211505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 772/1430] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:41 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:42.235428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-76692615-32d9-4468-a8dd-b43e595c97c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:42.236142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-76692615-32d9-4468-a8dd-b43e595c97c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:42.247796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76692615-32d9-4468-a8dd-b43e595c97c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:42.248769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76692615-32d9-4468-a8dd-b43e595c97c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:42.249555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76692615-32d9-4468-a8dd-b43e595c97c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:42.250859 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76692615-32d9-4468-a8dd-b43e595c97c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:42.251656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76692615-32d9-4468-a8dd-b43e595c97c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:42.252415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76692615-32d9-4468-a8dd-b43e595c97c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:42.341587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-76692615-32d9-4468-a8dd-b43e595c97c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:42.350742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-76692615-32d9-4468-a8dd-b43e595c97c3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.120101 Aug 07 01:23:42.351378 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 773/1431] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:42 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:43.502287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2fcaea06-1c22-474f-97e1-25b8d9e71097 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:43.503309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2fcaea06-1c22-474f-97e1-25b8d9e71097 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:43.510023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fcaea06-1c22-474f-97e1-25b8d9e71097 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:43.510661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fcaea06-1c22-474f-97e1-25b8d9e71097 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:43.511191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fcaea06-1c22-474f-97e1-25b8d9e71097 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:43.511982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fcaea06-1c22-474f-97e1-25b8d9e71097 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:43.512542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fcaea06-1c22-474f-97e1-25b8d9e71097 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:43.513057 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fcaea06-1c22-474f-97e1-25b8d9e71097 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:43.583513 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2fcaea06-1c22-474f-97e1-25b8d9e71097 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:43.592983 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2fcaea06-1c22-474f-97e1-25b8d9e71097 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.223623 Aug 07 01:23:43.593683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 774/1432] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:43 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:44.620573 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-44179642-ea9b-4dfe-82bf-8739162697aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:44.621441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-44179642-ea9b-4dfe-82bf-8739162697aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:44.633441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44179642-ea9b-4dfe-82bf-8739162697aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:44.634325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44179642-ea9b-4dfe-82bf-8739162697aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:44.635101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44179642-ea9b-4dfe-82bf-8739162697aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:44.636414 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44179642-ea9b-4dfe-82bf-8739162697aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:44.637256 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44179642-ea9b-4dfe-82bf-8739162697aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:44.638036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44179642-ea9b-4dfe-82bf-8739162697aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:44.739682 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-44179642-ea9b-4dfe-82bf-8739162697aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:44.748776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-44179642-ea9b-4dfe-82bf-8739162697aa tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.135762 Aug 07 01:23:44.749393 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 775/1433] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:44 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:45.775723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-37bcc6a7-6d1f-4950-ad01-319da60e1858 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:45.776969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-37bcc6a7-6d1f-4950-ad01-319da60e1858 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:45.789491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37bcc6a7-6d1f-4950-ad01-319da60e1858 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:45.790794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37bcc6a7-6d1f-4950-ad01-319da60e1858 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:45.791583 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37bcc6a7-6d1f-4950-ad01-319da60e1858 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:45.793092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37bcc6a7-6d1f-4950-ad01-319da60e1858 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:45.793917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37bcc6a7-6d1f-4950-ad01-319da60e1858 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:45.794643 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37bcc6a7-6d1f-4950-ad01-319da60e1858 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:45.915940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-37bcc6a7-6d1f-4950-ad01-319da60e1858 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:45.932362 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-37bcc6a7-6d1f-4950-ad01-319da60e1858 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.163712 Aug 07 01:23:45.933347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 776/1434] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:45 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:46.955670 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a6316070-fbe7-43fc-864f-6092b7519ee9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:46.956778 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a6316070-fbe7-43fc-864f-6092b7519ee9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:46.964613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6316070-fbe7-43fc-864f-6092b7519ee9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:46.965116 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6316070-fbe7-43fc-864f-6092b7519ee9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:46.965555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6316070-fbe7-43fc-864f-6092b7519ee9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:46.966281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6316070-fbe7-43fc-864f-6092b7519ee9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:46.966848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6316070-fbe7-43fc-864f-6092b7519ee9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:46.967335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6316070-fbe7-43fc-864f-6092b7519ee9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:47.053470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a6316070-fbe7-43fc-864f-6092b7519ee9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:47.068757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a6316070-fbe7-43fc-864f-6092b7519ee9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.116840 Aug 07 01:23:47.069796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 777/1435] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:46 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:48.103978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-98c30958-8296-4f67-a581-1a95f89263f6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:48.104741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-98c30958-8296-4f67-a581-1a95f89263f6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:48.113061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98c30958-8296-4f67-a581-1a95f89263f6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:48.113634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98c30958-8296-4f67-a581-1a95f89263f6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:48.114081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98c30958-8296-4f67-a581-1a95f89263f6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:48.114835 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98c30958-8296-4f67-a581-1a95f89263f6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:48.115298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98c30958-8296-4f67-a581-1a95f89263f6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:48.115731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98c30958-8296-4f67-a581-1a95f89263f6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:48.187001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-98c30958-8296-4f67-a581-1a95f89263f6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:48.200012 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-98c30958-8296-4f67-a581-1a95f89263f6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100178 Aug 07 01:23:48.201046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 778/1436] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:48 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:48.401220 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:23:48.402237 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:23:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518 (ip 10.222.0.26) !!! Aug 07 01:23:49.225411 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8f374b55-f673-4980-aad6-0a000b58f13b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:49.226116 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8f374b55-f673-4980-aad6-0a000b58f13b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:49.234470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f374b55-f673-4980-aad6-0a000b58f13b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:49.235028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f374b55-f673-4980-aad6-0a000b58f13b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:49.235510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f374b55-f673-4980-aad6-0a000b58f13b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:49.236324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f374b55-f673-4980-aad6-0a000b58f13b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:49.236840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f374b55-f673-4980-aad6-0a000b58f13b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:49.237381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f374b55-f673-4980-aad6-0a000b58f13b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:49.309709 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8f374b55-f673-4980-aad6-0a000b58f13b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:49.318497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8f374b55-f673-4980-aad6-0a000b58f13b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100080 Aug 07 01:23:49.319094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 779/1437] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:49 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:50.341205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-424afccf-539a-40af-a9d7-e65880e12122 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:50.342856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-424afccf-539a-40af-a9d7-e65880e12122 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:50.350482 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-424afccf-539a-40af-a9d7-e65880e12122 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:50.351252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-424afccf-539a-40af-a9d7-e65880e12122 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:50.351827 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-424afccf-539a-40af-a9d7-e65880e12122 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:50.352687 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-424afccf-539a-40af-a9d7-e65880e12122 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:50.353198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-424afccf-539a-40af-a9d7-e65880e12122 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:50.353666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-424afccf-539a-40af-a9d7-e65880e12122 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:50.459276 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-424afccf-539a-40af-a9d7-e65880e12122 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:50.468637 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-424afccf-539a-40af-a9d7-e65880e12122 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.131267 Aug 07 01:23:50.469042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 780/1438] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:50 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:51.493249 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f7387bcb-7950-420a-8457-c284e3e27d59 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:51.494344 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f7387bcb-7950-420a-8457-c284e3e27d59 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:51.506080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f7387bcb-7950-420a-8457-c284e3e27d59 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:51.506965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f7387bcb-7950-420a-8457-c284e3e27d59 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:51.507757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f7387bcb-7950-420a-8457-c284e3e27d59 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:51.509102 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f7387bcb-7950-420a-8457-c284e3e27d59 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:51.510098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f7387bcb-7950-420a-8457-c284e3e27d59 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:51.510876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f7387bcb-7950-420a-8457-c284e3e27d59 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:51.628709 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f7387bcb-7950-420a-8457-c284e3e27d59 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:51.638675 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f7387bcb-7950-420a-8457-c284e3e27d59 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.152205 Aug 07 01:23:51.639404 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 781/1439] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:51 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:52.678455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e6527239-7cae-4f3b-bead-6fb21e83ac2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:52.679493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e6527239-7cae-4f3b-bead-6fb21e83ac2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:52.691599 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6527239-7cae-4f3b-bead-6fb21e83ac2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:52.692492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6527239-7cae-4f3b-bead-6fb21e83ac2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:52.693275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6527239-7cae-4f3b-bead-6fb21e83ac2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:52.694552 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6527239-7cae-4f3b-bead-6fb21e83ac2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:52.695388 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6527239-7cae-4f3b-bead-6fb21e83ac2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:52.696215 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6527239-7cae-4f3b-bead-6fb21e83ac2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:52.793082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e6527239-7cae-4f3b-bead-6fb21e83ac2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:52.807199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e6527239-7cae-4f3b-bead-6fb21e83ac2a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.135890 Aug 07 01:23:52.808299 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 782/1440] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:52 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:53.835048 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-eb071ba6-32ff-49a6-b5d2-bcd553930631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:53.836062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-eb071ba6-32ff-49a6-b5d2-bcd553930631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:53.849347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb071ba6-32ff-49a6-b5d2-bcd553930631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:53.850260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb071ba6-32ff-49a6-b5d2-bcd553930631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:53.851060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb071ba6-32ff-49a6-b5d2-bcd553930631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:53.852418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb071ba6-32ff-49a6-b5d2-bcd553930631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:53.853254 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb071ba6-32ff-49a6-b5d2-bcd553930631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:53.854040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb071ba6-32ff-49a6-b5d2-bcd553930631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:53.961646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-eb071ba6-32ff-49a6-b5d2-bcd553930631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:53.971992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-eb071ba6-32ff-49a6-b5d2-bcd553930631 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.143553 Aug 07 01:23:53.973143 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 783/1441] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:53 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:23:55.007943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f2ecb1e1-a123-4de3-bb99-a33495fd444b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:55.008851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f2ecb1e1-a123-4de3-bb99-a33495fd444b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:55.016829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2ecb1e1-a123-4de3-bb99-a33495fd444b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:55.017267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2ecb1e1-a123-4de3-bb99-a33495fd444b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:55.017711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2ecb1e1-a123-4de3-bb99-a33495fd444b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:55.018444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2ecb1e1-a123-4de3-bb99-a33495fd444b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:55.018901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2ecb1e1-a123-4de3-bb99-a33495fd444b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:55.019335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2ecb1e1-a123-4de3-bb99-a33495fd444b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:55.096013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f2ecb1e1-a123-4de3-bb99-a33495fd444b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:55.105151 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f2ecb1e1-a123-4de3-bb99-a33495fd444b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.101456 Aug 07 01:23:55.105761 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 784/1442] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:55 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:56.127730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7bf56216-f991-4497-a8b5-88f4caeaaaff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:56.128572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7bf56216-f991-4497-a8b5-88f4caeaaaff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:56.137719 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bf56216-f991-4497-a8b5-88f4caeaaaff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:56.138925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bf56216-f991-4497-a8b5-88f4caeaaaff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:56.139416 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bf56216-f991-4497-a8b5-88f4caeaaaff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:56.140201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bf56216-f991-4497-a8b5-88f4caeaaaff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:56.140755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bf56216-f991-4497-a8b5-88f4caeaaaff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:56.141242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bf56216-f991-4497-a8b5-88f4caeaaaff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:56.212055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7bf56216-f991-4497-a8b5-88f4caeaaaff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:56.221124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7bf56216-f991-4497-a8b5-88f4caeaaaff tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.097679 Aug 07 01:23:56.221725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 785/1443] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:56 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:57.263972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-625f2ae2-a004-4300-a46d-b44387a4b9ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:57.264924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-625f2ae2-a004-4300-a46d-b44387a4b9ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:57.271875 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-625f2ae2-a004-4300-a46d-b44387a4b9ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:57.272488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-625f2ae2-a004-4300-a46d-b44387a4b9ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:57.272997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-625f2ae2-a004-4300-a46d-b44387a4b9ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:57.273819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-625f2ae2-a004-4300-a46d-b44387a4b9ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:57.274343 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-625f2ae2-a004-4300-a46d-b44387a4b9ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:57.274961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-625f2ae2-a004-4300-a46d-b44387a4b9ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:57.360786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-625f2ae2-a004-4300-a46d-b44387a4b9ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:57.375211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-625f2ae2-a004-4300-a46d-b44387a4b9ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.115371 Aug 07 01:23:57.376298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 786/1444] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:57 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:58.396933 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-73c53f66-bc98-4f1f-aa8b-1db2b0f75a91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:58.398021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-73c53f66-bc98-4f1f-aa8b-1db2b0f75a91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:58.405761 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73c53f66-bc98-4f1f-aa8b-1db2b0f75a91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:58.406582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73c53f66-bc98-4f1f-aa8b-1db2b0f75a91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:58.407106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73c53f66-bc98-4f1f-aa8b-1db2b0f75a91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:58.407857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73c53f66-bc98-4f1f-aa8b-1db2b0f75a91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:58.408356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73c53f66-bc98-4f1f-aa8b-1db2b0f75a91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:58.408787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73c53f66-bc98-4f1f-aa8b-1db2b0f75a91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:58.491857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-73c53f66-bc98-4f1f-aa8b-1db2b0f75a91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:58.500061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-73c53f66-bc98-4f1f-aa8b-1db2b0f75a91 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.107002 Aug 07 01:23:58.500698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 787/1445] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:58 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:23:59.521801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6df7d650-dd59-4998-940b-ef4e45c6e77b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:23:59.522548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6df7d650-dd59-4998-940b-ef4e45c6e77b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:23:59.530907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6df7d650-dd59-4998-940b-ef4e45c6e77b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:59.531495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6df7d650-dd59-4998-940b-ef4e45c6e77b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:59.531889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6df7d650-dd59-4998-940b-ef4e45c6e77b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:59.532675 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6df7d650-dd59-4998-940b-ef4e45c6e77b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:23:59.533255 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6df7d650-dd59-4998-940b-ef4e45c6e77b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:23:59.533707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6df7d650-dd59-4998-940b-ef4e45c6e77b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:23:59.619112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6df7d650-dd59-4998-940b-ef4e45c6e77b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:23:59.625487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:23:59.635233 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6df7d650-dd59-4998-940b-ef4e45c6e77b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.117365 Aug 07 01:23:59.636268 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 788/1446] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:23:59 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:24:00.672831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-766636df-ebcb-43c3-b382-c8d13f4126e2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:24:00.673600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-766636df-ebcb-43c3-b382-c8d13f4126e2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:24:00.680378 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-766636df-ebcb-43c3-b382-c8d13f4126e2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:24:00.680916 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-766636df-ebcb-43c3-b382-c8d13f4126e2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:24:00.681328 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-766636df-ebcb-43c3-b382-c8d13f4126e2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:24:00.682066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-766636df-ebcb-43c3-b382-c8d13f4126e2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:24:00.682561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-766636df-ebcb-43c3-b382-c8d13f4126e2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:24:00.682960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-766636df-ebcb-43c3-b382-c8d13f4126e2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:24:00.755764 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-766636df-ebcb-43c3-b382-c8d13f4126e2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:24:00.770873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-766636df-ebcb-43c3-b382-c8d13f4126e2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.103778 Aug 07 01:24:00.771471 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 789/1447] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:24:00 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:24:01.792502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c9700606-1de4-4f03-8e51-3fc4bbb45cc7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:24:01.793563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c9700606-1de4-4f03-8e51-3fc4bbb45cc7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:24:01.802060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c9700606-1de4-4f03-8e51-3fc4bbb45cc7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:24:01.802561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c9700606-1de4-4f03-8e51-3fc4bbb45cc7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:24:01.803007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c9700606-1de4-4f03-8e51-3fc4bbb45cc7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:24:01.803741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c9700606-1de4-4f03-8e51-3fc4bbb45cc7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:24:01.804202 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c9700606-1de4-4f03-8e51-3fc4bbb45cc7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:24:01.804683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c9700606-1de4-4f03-8e51-3fc4bbb45cc7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:24:01.883212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c9700606-1de4-4f03-8e51-3fc4bbb45cc7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:24:01.891575 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c9700606-1de4-4f03-8e51-3fc4bbb45cc7 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.103014 Aug 07 01:24:01.892180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 790/1448] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:24:01 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:24:02.914498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d1239b65-a78e-4ee5-ba2a-a1be1e3eb0f9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:24:02.915278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d1239b65-a78e-4ee5-ba2a-a1be1e3eb0f9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:24:02.922894 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1239b65-a78e-4ee5-ba2a-a1be1e3eb0f9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:24:02.923662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1239b65-a78e-4ee5-ba2a-a1be1e3eb0f9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:24:02.924032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1239b65-a78e-4ee5-ba2a-a1be1e3eb0f9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:24:02.924872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1239b65-a78e-4ee5-ba2a-a1be1e3eb0f9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:24:02.925397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1239b65-a78e-4ee5-ba2a-a1be1e3eb0f9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:24:02.925882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1239b65-a78e-4ee5-ba2a-a1be1e3eb0f9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:24:03.000676 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d1239b65-a78e-4ee5-ba2a-a1be1e3eb0f9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:24:03.015236 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d1239b65-a78e-4ee5-ba2a-a1be1e3eb0f9 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.105295 Aug 07 01:24:03.016107 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 791/1449] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:24:02 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:24:04.051827 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-704b20f6-d9c8-46ff-b560-097cd1ce4547 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:24:04.052979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-704b20f6-d9c8-46ff-b560-097cd1ce4547 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:24:04.060283 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-704b20f6-d9c8-46ff-b560-097cd1ce4547 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:24:04.060814 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-704b20f6-d9c8-46ff-b560-097cd1ce4547 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:24:04.061260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-704b20f6-d9c8-46ff-b560-097cd1ce4547 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:24:04.061981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-704b20f6-d9c8-46ff-b560-097cd1ce4547 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:24:04.062434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-704b20f6-d9c8-46ff-b560-097cd1ce4547 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:24:04.062862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-704b20f6-d9c8-46ff-b560-097cd1ce4547 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:24:04.154493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-704b20f6-d9c8-46ff-b560-097cd1ce4547 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:24:04.162651 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-704b20f6-d9c8-46ff-b560-097cd1ce4547 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.115563 Aug 07 01:24:04.163243 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 792/1450] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:24:04 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:24:04.461645 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1c349ddd-c33e-4a3b-b6a1-2303f89f54ad tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:24:04.462906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1c349ddd-c33e-4a3b-b6a1-2303f89f54ad tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:24:04.476074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c349ddd-c33e-4a3b-b6a1-2303f89f54ad tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:24:04.476978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c349ddd-c33e-4a3b-b6a1-2303f89f54ad tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:24:04.477793 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c349ddd-c33e-4a3b-b6a1-2303f89f54ad tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:24:04.479093 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c349ddd-c33e-4a3b-b6a1-2303f89f54ad tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:24:04.479939 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c349ddd-c33e-4a3b-b6a1-2303f89f54ad tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:24:04.480755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c349ddd-c33e-4a3b-b6a1-2303f89f54ad tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:24:14.469632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=49 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:24:14.526826 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=28 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:25:04.425697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c7be3fee-cb75-40d1-9094-90ccc1c38d47 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518" status: 202 len: 0 microversion: 2.1 time: 119.974747 Aug 07 01:25:04.426692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 659/1451] 10.222.0.26 () {58 vars in 1416 bytes} [Sun Aug 7 01:23:04 2022] DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518 => generated 0 bytes in 119977 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:25:04.437074 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cbbf45f5-8fb6-448f-865e-899d8a6cafcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:04.438353 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-cbbf45f5-8fb6-448f-865e-899d8a6cafcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:04.451999 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbbf45f5-8fb6-448f-865e-899d8a6cafcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:04.452919 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbbf45f5-8fb6-448f-865e-899d8a6cafcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:04.453962 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbbf45f5-8fb6-448f-865e-899d8a6cafcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:04.455172 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbbf45f5-8fb6-448f-865e-899d8a6cafcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:04.456337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbbf45f5-8fb6-448f-865e-899d8a6cafcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:04.457050 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbbf45f5-8fb6-448f-865e-899d8a6cafcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:04.596500 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-cbbf45f5-8fb6-448f-865e-899d8a6cafcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:04.611406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-cbbf45f5-8fb6-448f-865e-899d8a6cafcc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.182599 Aug 07 01:25:04.612585 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 660/1452] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:04 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:25:04.621984 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-dab7a9d0-38fe-4315-b164-430b2c43b432 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:04.622504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-dab7a9d0-38fe-4315-b164-430b2c43b432 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:04.634441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dab7a9d0-38fe-4315-b164-430b2c43b432 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:04.635343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dab7a9d0-38fe-4315-b164-430b2c43b432 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:04.636770 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dab7a9d0-38fe-4315-b164-430b2c43b432 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:04.638392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dab7a9d0-38fe-4315-b164-430b2c43b432 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:04.639278 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dab7a9d0-38fe-4315-b164-430b2c43b432 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:04.640067 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dab7a9d0-38fe-4315-b164-430b2c43b432 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:04.685203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-dab7a9d0-38fe-4315-b164-430b2c43b432 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be/ips" status: 200 len: 150 microversion: 2.1 time: 0.070994 Aug 07 01:25:04.685855 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 661/1453] 10.222.0.26 () {58 vars in 1248 bytes} [Sun Aug 7 01:25:04 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be/ips => generated 150 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:25:04.691807 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-19810c6d-4632-48de-b6f4-6b2df70259e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:04.692457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-19810c6d-4632-48de-b6f4-6b2df70259e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:04.701501 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19810c6d-4632-48de-b6f4-6b2df70259e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:04.701821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19810c6d-4632-48de-b6f4-6b2df70259e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:04.702387 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19810c6d-4632-48de-b6f4-6b2df70259e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:04.703102 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19810c6d-4632-48de-b6f4-6b2df70259e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:04.703572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19810c6d-4632-48de-b6f4-6b2df70259e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:04.704005 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19810c6d-4632-48de-b6f4-6b2df70259e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:04.805295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.objects.instance [None req-1c349ddd-c33e-4a3b-b6a1-2303f89f54ad tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lazy-loading 'services' on Instance uuid f481f4cc-2803-4f12-9cd2-db6055d1488c {{(pid=117983) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:25:14.189596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:25:14.190345 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:25:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518 (ip 10.222.0.26) !!! Aug 07 01:25:14.698163 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=6 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:25:14.730472 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=15,nova_cell1:INSERT=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:25:14.813819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=4 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:25:17.670342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.objects.instance [None req-19810c6d-4632-48de-b6f4-6b2df70259e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lazy-loading 'services' on Instance uuid f481f4cc-2803-4f12-9cd2-db6055d1488c {{(pid=117984) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:25:17.735672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-1c349ddd-c33e-4a3b-b6a1-2303f89f54ad tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-56a97bd9-eb2c-4ee1-b075-4e5cf91c1da4) Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1c349ddd-c33e-4a3b-b6a1-2303f89f54ad tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Request method failure captured: Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518 HTTP/1.1 Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xGwdVHZVWnHIzGDSpTVNiz-q980RAQtQDxjuRjGuqs_HJoXOXoBhZ0fF1bbTTjk8m9Pvp2IcKkTppafVNQBMvnpaEQe9WwCTTgrFUCyEMZcpdieiPEfKYhmeOU5srzGG4O4YeIgb7xO3BdRG6OZeDumpyuz3KQXJRzGLfuf1xxtaZM Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: f7b08cc6d4a34961bdf464b58946a02e Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:25:17.740486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:25:17.743439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:25:17.743439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:25:17.743439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: f7b08cc6d4a34961bdf464b58946a02e Aug 07 01:25:17.743439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:25:17.743439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-VolumesSnapshotTestJSON-2086260047-project Aug 07 01:25:17.743439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:25:17.743439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:25:17.743439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 6dd0ed2144164190b380ed58b3dd7cd4 Aug 07 01:25:17.743439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-VolumesSnapshotTestJSON-2086260047-project Aug 07 01:25:17.743439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-56a97bd9-eb2c-4ee1-b075-4e5cf91c1da4) Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'server_id': 'f481f4cc-2803-4f12-9cd2-db6055d1488c', 'id': '830234fe-bbd1-4781-97ce-8e340a8d9518'} Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 07 01:25:17.745542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-56a97bd9-eb2c-4ee1-b075-4e5cf91c1da4) Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in _detach_volume Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:25:17.748691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-56a97bd9-eb2c-4ee1-b075-4e5cf91c1da4) Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:25:17.751780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5152, in detach_volume Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5132, in _detach_volume Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-56a97bd9-eb2c-4ee1-b075-4e5cf91c1da4) Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:25:17.753845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-56a97bd9-eb2c-4ee1-b075-4e5cf91c1da4) Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-56a97bd9-eb2c-4ee1-b075-4e5cf91c1da4) Aug 07 01:25:17.756035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:25:17.758533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1c349ddd-c33e-4a3b-b6a1-2303f89f54ad tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-56a97bd9-eb2c-4ee1-b075-4e5cf91c1da4) {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:25:17.758533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1c349ddd-c33e-4a3b-b6a1-2303f89f54ad tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518" status: 400 len: 269 microversion: 2.1 time: 73.443807 Aug 07 01:25:17.758533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 793/1454] 10.222.0.26 () {58 vars in 1416 bytes} [Sun Aug 7 01:24:04 2022] DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518 => generated 269 bytes in 73447 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Aug 07 01:25:17.758533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6f9c5700-ae6e-4e6d-ac54-e1bed35ec78f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:17.759500 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6f9c5700-ae6e-4e6d-ac54-e1bed35ec78f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:17.774707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f9c5700-ae6e-4e6d-ac54-e1bed35ec78f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:17.775649 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f9c5700-ae6e-4e6d-ac54-e1bed35ec78f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:17.776591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f9c5700-ae6e-4e6d-ac54-e1bed35ec78f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:17.778960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f9c5700-ae6e-4e6d-ac54-e1bed35ec78f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:17.780122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f9c5700-ae6e-4e6d-ac54-e1bed35ec78f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:17.781230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f9c5700-ae6e-4e6d-ac54-e1bed35ec78f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:17.892854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6f9c5700-ae6e-4e6d-ac54-e1bed35ec78f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:17.905098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6f9c5700-ae6e-4e6d-ac54-e1bed35ec78f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.157984 Aug 07 01:25:17.905708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 794/1455] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:17 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:25:18.946062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-09d21604-472e-42fd-ad7e-02df07b11b44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:18.947232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-09d21604-472e-42fd-ad7e-02df07b11b44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:18.961919 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09d21604-472e-42fd-ad7e-02df07b11b44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:18.962787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09d21604-472e-42fd-ad7e-02df07b11b44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:18.963572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09d21604-472e-42fd-ad7e-02df07b11b44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:18.964907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09d21604-472e-42fd-ad7e-02df07b11b44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:18.966394 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09d21604-472e-42fd-ad7e-02df07b11b44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:18.967722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09d21604-472e-42fd-ad7e-02df07b11b44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:19.086197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-09d21604-472e-42fd-ad7e-02df07b11b44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:19.102622 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-09d21604-472e-42fd-ad7e-02df07b11b44 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.164154 Aug 07 01:25:19.103598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 795/1456] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:18 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:20.140285 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-91ac7d25-a9f6-4f8c-8824-73cfebc33f33 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:20.141329 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-91ac7d25-a9f6-4f8c-8824-73cfebc33f33 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:20.149676 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91ac7d25-a9f6-4f8c-8824-73cfebc33f33 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:20.150179 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91ac7d25-a9f6-4f8c-8824-73cfebc33f33 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:20.150623 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91ac7d25-a9f6-4f8c-8824-73cfebc33f33 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:20.151494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91ac7d25-a9f6-4f8c-8824-73cfebc33f33 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:20.152086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91ac7d25-a9f6-4f8c-8824-73cfebc33f33 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:20.152553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91ac7d25-a9f6-4f8c-8824-73cfebc33f33 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:20.265014 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-91ac7d25-a9f6-4f8c-8824-73cfebc33f33 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:20.279292 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-91ac7d25-a9f6-4f8c-8824-73cfebc33f33 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.144256 Aug 07 01:25:20.280446 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 796/1457] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:20 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:21.320558 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cec42e0e-91aa-4d0c-9ba6-d9d9c412c07c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:21.321666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cec42e0e-91aa-4d0c-9ba6-d9d9c412c07c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:21.334720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cec42e0e-91aa-4d0c-9ba6-d9d9c412c07c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:21.335604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cec42e0e-91aa-4d0c-9ba6-d9d9c412c07c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:21.336450 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cec42e0e-91aa-4d0c-9ba6-d9d9c412c07c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:21.337786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cec42e0e-91aa-4d0c-9ba6-d9d9c412c07c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:21.339188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cec42e0e-91aa-4d0c-9ba6-d9d9c412c07c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:21.339979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cec42e0e-91aa-4d0c-9ba6-d9d9c412c07c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:21.429462 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-cec42e0e-91aa-4d0c-9ba6-d9d9c412c07c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:21.440951 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cec42e0e-91aa-4d0c-9ba6-d9d9c412c07c tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.127662 Aug 07 01:25:21.441723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 797/1458] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:21 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:22.464418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7a061b5f-6532-4c8e-adbf-47c9f2dc78a6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:22.465702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7a061b5f-6532-4c8e-adbf-47c9f2dc78a6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:22.477416 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7a061b5f-6532-4c8e-adbf-47c9f2dc78a6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:22.478305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7a061b5f-6532-4c8e-adbf-47c9f2dc78a6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:22.479085 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7a061b5f-6532-4c8e-adbf-47c9f2dc78a6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:22.480058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7a061b5f-6532-4c8e-adbf-47c9f2dc78a6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:22.480625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7a061b5f-6532-4c8e-adbf-47c9f2dc78a6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:22.481411 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7a061b5f-6532-4c8e-adbf-47c9f2dc78a6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:22.593783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7a061b5f-6532-4c8e-adbf-47c9f2dc78a6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:22.608443 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7a061b5f-6532-4c8e-adbf-47c9f2dc78a6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.147873 Aug 07 01:25:22.609519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 798/1459] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:22 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:23.643392 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c98db268-6e35-49a4-92d5-7cf148ee2c1f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:23.644181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c98db268-6e35-49a4-92d5-7cf148ee2c1f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:23.652157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c98db268-6e35-49a4-92d5-7cf148ee2c1f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:23.652844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c98db268-6e35-49a4-92d5-7cf148ee2c1f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:23.653348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c98db268-6e35-49a4-92d5-7cf148ee2c1f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:23.654217 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c98db268-6e35-49a4-92d5-7cf148ee2c1f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:23.654657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c98db268-6e35-49a4-92d5-7cf148ee2c1f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:23.655686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c98db268-6e35-49a4-92d5-7cf148ee2c1f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:23.738816 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c98db268-6e35-49a4-92d5-7cf148ee2c1f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:23.748522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c98db268-6e35-49a4-92d5-7cf148ee2c1f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.109636 Aug 07 01:25:23.749212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 799/1460] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:23 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:24.496747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:25:24.497053 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:25:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:25:24.777757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7574cacf-d9ff-49f7-a744-84b29f1645c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:24.778926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7574cacf-d9ff-49f7-a744-84b29f1645c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:24.790807 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7574cacf-d9ff-49f7-a744-84b29f1645c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:24.792199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7574cacf-d9ff-49f7-a744-84b29f1645c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:24.793027 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7574cacf-d9ff-49f7-a744-84b29f1645c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:24.794650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7574cacf-d9ff-49f7-a744-84b29f1645c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:24.795467 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7574cacf-d9ff-49f7-a744-84b29f1645c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:24.796665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7574cacf-d9ff-49f7-a744-84b29f1645c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:24.921744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7574cacf-d9ff-49f7-a744-84b29f1645c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:24.935531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7574cacf-d9ff-49f7-a744-84b29f1645c1 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.165055 Aug 07 01:25:24.936683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 800/1461] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:24 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:25.975742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3d48114b-69d6-4017-9568-8120ba9d43ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:25.976910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3d48114b-69d6-4017-9568-8120ba9d43ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:25.990118 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d48114b-69d6-4017-9568-8120ba9d43ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:25.990904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d48114b-69d6-4017-9568-8120ba9d43ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:25.991674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d48114b-69d6-4017-9568-8120ba9d43ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:25.993033 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d48114b-69d6-4017-9568-8120ba9d43ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:25.993890 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d48114b-69d6-4017-9568-8120ba9d43ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:25.995178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d48114b-69d6-4017-9568-8120ba9d43ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:26.117851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3d48114b-69d6-4017-9568-8120ba9d43ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:26.128091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3d48114b-69d6-4017-9568-8120ba9d43ab tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.159888 Aug 07 01:25:26.128776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 801/1462] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:25 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:27.153645 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-942f4cee-20f0-4247-8959-50ee3d63ed71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:27.154396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-942f4cee-20f0-4247-8959-50ee3d63ed71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:27.162180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-942f4cee-20f0-4247-8959-50ee3d63ed71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:27.162684 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-942f4cee-20f0-4247-8959-50ee3d63ed71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:27.163119 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-942f4cee-20f0-4247-8959-50ee3d63ed71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:27.163913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-942f4cee-20f0-4247-8959-50ee3d63ed71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:27.164410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-942f4cee-20f0-4247-8959-50ee3d63ed71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:27.164844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-942f4cee-20f0-4247-8959-50ee3d63ed71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:27.238354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-942f4cee-20f0-4247-8959-50ee3d63ed71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:27.247703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-942f4cee-20f0-4247-8959-50ee3d63ed71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.098021 Aug 07 01:25:27.248342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 802/1463] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:27 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:27.693383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=4 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:25:28.278813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7cdecdb3-a99c-4b78-8014-df44b3de4031 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:28.279567 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7cdecdb3-a99c-4b78-8014-df44b3de4031 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:28.287238 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7cdecdb3-a99c-4b78-8014-df44b3de4031 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:28.287774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7cdecdb3-a99c-4b78-8014-df44b3de4031 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:28.288230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7cdecdb3-a99c-4b78-8014-df44b3de4031 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:28.289037 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7cdecdb3-a99c-4b78-8014-df44b3de4031 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:28.289763 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7cdecdb3-a99c-4b78-8014-df44b3de4031 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:28.290207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7cdecdb3-a99c-4b78-8014-df44b3de4031 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:28.361586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7cdecdb3-a99c-4b78-8014-df44b3de4031 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:28.370303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7cdecdb3-a99c-4b78-8014-df44b3de4031 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.095715 Aug 07 01:25:28.370896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 803/1464] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:28 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:29.396050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9a83736b-e95f-4549-9ea3-e27d8bffe061 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:29.397257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9a83736b-e95f-4549-9ea3-e27d8bffe061 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:29.409408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a83736b-e95f-4549-9ea3-e27d8bffe061 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:29.410274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a83736b-e95f-4549-9ea3-e27d8bffe061 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:29.410965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a83736b-e95f-4549-9ea3-e27d8bffe061 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:29.412323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a83736b-e95f-4549-9ea3-e27d8bffe061 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:29.413169 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a83736b-e95f-4549-9ea3-e27d8bffe061 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:29.413960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a83736b-e95f-4549-9ea3-e27d8bffe061 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:29.514270 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9a83736b-e95f-4549-9ea3-e27d8bffe061 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:29.528787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9a83736b-e95f-4549-9ea3-e27d8bffe061 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.139596 Aug 07 01:25:29.529904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 804/1465] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:29 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:25:30.551812 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a0ad1385-e816-4201-b1ed-adc34ca947e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:30.552657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a0ad1385-e816-4201-b1ed-adc34ca947e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:30.560223 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0ad1385-e816-4201-b1ed-adc34ca947e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:30.560768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0ad1385-e816-4201-b1ed-adc34ca947e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:30.561278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0ad1385-e816-4201-b1ed-adc34ca947e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:30.562193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0ad1385-e816-4201-b1ed-adc34ca947e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:30.562666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0ad1385-e816-4201-b1ed-adc34ca947e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:30.563106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a0ad1385-e816-4201-b1ed-adc34ca947e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:30.635587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a0ad1385-e816-4201-b1ed-adc34ca947e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:30.644401 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a0ad1385-e816-4201-b1ed-adc34ca947e5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.096737 Aug 07 01:25:30.645006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 805/1466] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:30 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:31.666531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-32b83fb4-cc58-4c8d-89cb-9cd61a178738 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:31.667565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-32b83fb4-cc58-4c8d-89cb-9cd61a178738 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:31.677211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32b83fb4-cc58-4c8d-89cb-9cd61a178738 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:31.678088 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32b83fb4-cc58-4c8d-89cb-9cd61a178738 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:31.678903 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32b83fb4-cc58-4c8d-89cb-9cd61a178738 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:31.679696 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32b83fb4-cc58-4c8d-89cb-9cd61a178738 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:31.680166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32b83fb4-cc58-4c8d-89cb-9cd61a178738 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:31.680639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32b83fb4-cc58-4c8d-89cb-9cd61a178738 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:31.762402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-32b83fb4-cc58-4c8d-89cb-9cd61a178738 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:31.776399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-32b83fb4-cc58-4c8d-89cb-9cd61a178738 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.113297 Aug 07 01:25:31.777428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 806/1467] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:31 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:25:32.809771 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a993be2c-dbff-4d1e-89b0-60c915e52965 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:32.810705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a993be2c-dbff-4d1e-89b0-60c915e52965 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:32.818009 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a993be2c-dbff-4d1e-89b0-60c915e52965 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:32.818559 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a993be2c-dbff-4d1e-89b0-60c915e52965 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:32.819086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a993be2c-dbff-4d1e-89b0-60c915e52965 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:32.819938 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a993be2c-dbff-4d1e-89b0-60c915e52965 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:32.820508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a993be2c-dbff-4d1e-89b0-60c915e52965 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:32.820986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a993be2c-dbff-4d1e-89b0-60c915e52965 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:32.894736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a993be2c-dbff-4d1e-89b0-60c915e52965 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:32.909320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a993be2c-dbff-4d1e-89b0-60c915e52965 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.103342 Aug 07 01:25:32.910352 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 807/1468] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:32 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:33.933608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d6533dd3-f3df-4658-a89b-83735e318834 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:33.934744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d6533dd3-f3df-4658-a89b-83735e318834 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:33.942728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d6533dd3-f3df-4658-a89b-83735e318834 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:33.943252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d6533dd3-f3df-4658-a89b-83735e318834 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:33.943695 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d6533dd3-f3df-4658-a89b-83735e318834 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:33.944455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d6533dd3-f3df-4658-a89b-83735e318834 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:33.944918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d6533dd3-f3df-4658-a89b-83735e318834 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:33.945352 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d6533dd3-f3df-4658-a89b-83735e318834 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:34.024819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d6533dd3-f3df-4658-a89b-83735e318834 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:34.035115 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d6533dd3-f3df-4658-a89b-83735e318834 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.105921 Aug 07 01:25:34.035863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 808/1469] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:33 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:35.057878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-31d45bf2-8846-4942-a168-3fdf427d720f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:35.058727 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-31d45bf2-8846-4942-a168-3fdf427d720f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:35.066101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31d45bf2-8846-4942-a168-3fdf427d720f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:35.066611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31d45bf2-8846-4942-a168-3fdf427d720f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:35.067171 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31d45bf2-8846-4942-a168-3fdf427d720f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:35.067794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31d45bf2-8846-4942-a168-3fdf427d720f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:35.068280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31d45bf2-8846-4942-a168-3fdf427d720f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:35.068718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31d45bf2-8846-4942-a168-3fdf427d720f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:35.147077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-31d45bf2-8846-4942-a168-3fdf427d720f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:35.158742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-31d45bf2-8846-4942-a168-3fdf427d720f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.104615 Aug 07 01:25:35.159781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 809/1470] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:35 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:36.184101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-59530038-8304-43db-b650-5ee4221d7b25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:36.184998 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-59530038-8304-43db-b650-5ee4221d7b25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:36.192690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-59530038-8304-43db-b650-5ee4221d7b25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:36.193357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-59530038-8304-43db-b650-5ee4221d7b25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:36.193901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-59530038-8304-43db-b650-5ee4221d7b25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:36.194831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-59530038-8304-43db-b650-5ee4221d7b25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:36.195398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-59530038-8304-43db-b650-5ee4221d7b25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:36.195931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-59530038-8304-43db-b650-5ee4221d7b25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:36.219795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:25:36.273683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-59530038-8304-43db-b650-5ee4221d7b25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:36.283903 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-59530038-8304-43db-b650-5ee4221d7b25 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.104219 Aug 07 01:25:36.285031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 810/1471] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:36 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:37.306166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e4d74c19-8037-49e2-b669-f441e9c05f8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:37.307502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e4d74c19-8037-49e2-b669-f441e9c05f8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:37.321209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4d74c19-8037-49e2-b669-f441e9c05f8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:37.321810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4d74c19-8037-49e2-b669-f441e9c05f8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:37.323012 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4d74c19-8037-49e2-b669-f441e9c05f8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:37.324063 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4d74c19-8037-49e2-b669-f441e9c05f8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:37.324888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4d74c19-8037-49e2-b669-f441e9c05f8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:37.325366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4d74c19-8037-49e2-b669-f441e9c05f8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:37.394455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e4d74c19-8037-49e2-b669-f441e9c05f8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:37.402873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e4d74c19-8037-49e2-b669-f441e9c05f8b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100888 Aug 07 01:25:37.403584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 811/1472] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:37 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:38.430892 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-61d8e1f9-f9c2-4753-ae29-a0e2813adb80 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:38.432439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-61d8e1f9-f9c2-4753-ae29-a0e2813adb80 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:38.439530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61d8e1f9-f9c2-4753-ae29-a0e2813adb80 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:38.440029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61d8e1f9-f9c2-4753-ae29-a0e2813adb80 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:38.440520 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61d8e1f9-f9c2-4753-ae29-a0e2813adb80 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:38.441269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61d8e1f9-f9c2-4753-ae29-a0e2813adb80 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:38.441731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61d8e1f9-f9c2-4753-ae29-a0e2813adb80 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:38.442164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61d8e1f9-f9c2-4753-ae29-a0e2813adb80 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:38.519248 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-61d8e1f9-f9c2-4753-ae29-a0e2813adb80 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:38.534476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-61d8e1f9-f9c2-4753-ae29-a0e2813adb80 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.107407 Aug 07 01:25:38.535417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 812/1473] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:38 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:39.557710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2b078a8f-3faa-42d6-b822-0038c3285452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:39.558506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2b078a8f-3faa-42d6-b822-0038c3285452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:39.565206 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b078a8f-3faa-42d6-b822-0038c3285452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:39.565797 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b078a8f-3faa-42d6-b822-0038c3285452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:39.566395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b078a8f-3faa-42d6-b822-0038c3285452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:39.567162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b078a8f-3faa-42d6-b822-0038c3285452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:39.567651 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b078a8f-3faa-42d6-b822-0038c3285452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:39.568086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b078a8f-3faa-42d6-b822-0038c3285452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:39.652397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2b078a8f-3faa-42d6-b822-0038c3285452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:39.661227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2b078a8f-3faa-42d6-b822-0038c3285452 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.108490 Aug 07 01:25:39.662147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 813/1474] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:39 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:40.696353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d0918ecc-f9ce-4e74-83e8-fde2556658cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:40.697119 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d0918ecc-f9ce-4e74-83e8-fde2556658cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:40.704207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0918ecc-f9ce-4e74-83e8-fde2556658cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:40.704729 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0918ecc-f9ce-4e74-83e8-fde2556658cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:40.705166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0918ecc-f9ce-4e74-83e8-fde2556658cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:40.705899 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0918ecc-f9ce-4e74-83e8-fde2556658cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:40.706347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0918ecc-f9ce-4e74-83e8-fde2556658cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:40.706770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0918ecc-f9ce-4e74-83e8-fde2556658cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:40.778637 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d0918ecc-f9ce-4e74-83e8-fde2556658cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:40.787040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d0918ecc-f9ce-4e74-83e8-fde2556658cc tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.094716 Aug 07 01:25:40.787735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 814/1475] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:40 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:41.810058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-68e79a1f-759a-4fd6-b610-ecffed93101a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:41.810845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-68e79a1f-759a-4fd6-b610-ecffed93101a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:41.818261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68e79a1f-759a-4fd6-b610-ecffed93101a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:41.818800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68e79a1f-759a-4fd6-b610-ecffed93101a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:41.819210 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68e79a1f-759a-4fd6-b610-ecffed93101a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:41.819965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68e79a1f-759a-4fd6-b610-ecffed93101a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:41.820462 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68e79a1f-759a-4fd6-b610-ecffed93101a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:41.820894 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68e79a1f-759a-4fd6-b610-ecffed93101a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:41.903863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-68e79a1f-759a-4fd6-b610-ecffed93101a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:41.919077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-68e79a1f-759a-4fd6-b610-ecffed93101a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.113573 Aug 07 01:25:41.920189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 815/1476] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:41 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:42.954455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-29bfaa81-924b-46f0-824e-79c216883a0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:42.955251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-29bfaa81-924b-46f0-824e-79c216883a0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:42.964140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-29bfaa81-924b-46f0-824e-79c216883a0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:42.965052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-29bfaa81-924b-46f0-824e-79c216883a0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:42.965702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-29bfaa81-924b-46f0-824e-79c216883a0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:42.966863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-29bfaa81-924b-46f0-824e-79c216883a0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:42.967604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-29bfaa81-924b-46f0-824e-79c216883a0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:42.968581 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-29bfaa81-924b-46f0-824e-79c216883a0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:43.052599 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-29bfaa81-924b-46f0-824e-79c216883a0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:43.061907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-29bfaa81-924b-46f0-824e-79c216883a0d tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.111924 Aug 07 01:25:43.062529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 816/1477] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:42 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:44.084459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f20079d8-55da-46c6-b307-389ab51e93f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:44.085197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f20079d8-55da-46c6-b307-389ab51e93f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:44.092198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f20079d8-55da-46c6-b307-389ab51e93f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:44.092732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f20079d8-55da-46c6-b307-389ab51e93f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:44.093176 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f20079d8-55da-46c6-b307-389ab51e93f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:44.093927 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f20079d8-55da-46c6-b307-389ab51e93f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:44.094387 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f20079d8-55da-46c6-b307-389ab51e93f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:44.094810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f20079d8-55da-46c6-b307-389ab51e93f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:44.170765 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f20079d8-55da-46c6-b307-389ab51e93f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:44.185707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f20079d8-55da-46c6-b307-389ab51e93f8 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.104954 Aug 07 01:25:44.186766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 817/1478] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:44 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:45.220381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-05757562-d73d-4069-9a5d-424a47b834d3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:45.221268 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-05757562-d73d-4069-9a5d-424a47b834d3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:45.223508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=49 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:25:45.231039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05757562-d73d-4069-9a5d-424a47b834d3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:45.231039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05757562-d73d-4069-9a5d-424a47b834d3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:45.231039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05757562-d73d-4069-9a5d-424a47b834d3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:45.232608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05757562-d73d-4069-9a5d-424a47b834d3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:45.233076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05757562-d73d-4069-9a5d-424a47b834d3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:45.234007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-05757562-d73d-4069-9a5d-424a47b834d3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:45.307683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-05757562-d73d-4069-9a5d-424a47b834d3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:45.316003 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-05757562-d73d-4069-9a5d-424a47b834d3 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.099642 Aug 07 01:25:45.316986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 818/1479] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:45 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:46.345846 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d815b670-ceaf-4f12-8c23-83231fc0ad53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:46.346923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d815b670-ceaf-4f12-8c23-83231fc0ad53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:46.359726 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d815b670-ceaf-4f12-8c23-83231fc0ad53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:46.360680 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d815b670-ceaf-4f12-8c23-83231fc0ad53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:46.361488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d815b670-ceaf-4f12-8c23-83231fc0ad53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:46.363027 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d815b670-ceaf-4f12-8c23-83231fc0ad53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:46.364076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d815b670-ceaf-4f12-8c23-83231fc0ad53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:46.365174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d815b670-ceaf-4f12-8c23-83231fc0ad53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:46.469585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d815b670-ceaf-4f12-8c23-83231fc0ad53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:46.479693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d815b670-ceaf-4f12-8c23-83231fc0ad53 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.141695 Aug 07 01:25:46.480336 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 819/1480] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:46 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:47.502841 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4206adf7-c378-44d1-a7b8-ed3a47a53900 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:47.503810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4206adf7-c378-44d1-a7b8-ed3a47a53900 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:47.511284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4206adf7-c378-44d1-a7b8-ed3a47a53900 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:47.511774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4206adf7-c378-44d1-a7b8-ed3a47a53900 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:47.512210 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4206adf7-c378-44d1-a7b8-ed3a47a53900 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:47.512960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4206adf7-c378-44d1-a7b8-ed3a47a53900 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:47.513417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4206adf7-c378-44d1-a7b8-ed3a47a53900 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:47.513886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4206adf7-c378-44d1-a7b8-ed3a47a53900 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:47.589751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4206adf7-c378-44d1-a7b8-ed3a47a53900 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:47.598340 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4206adf7-c378-44d1-a7b8-ed3a47a53900 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.099333 Aug 07 01:25:47.598934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 820/1481] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:47 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:48.620075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ab401244-aa05-4818-b565-094d2d22afc5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:48.620895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ab401244-aa05-4818-b565-094d2d22afc5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:48.628079 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab401244-aa05-4818-b565-094d2d22afc5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:48.628608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab401244-aa05-4818-b565-094d2d22afc5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:48.629053 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab401244-aa05-4818-b565-094d2d22afc5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:48.629762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab401244-aa05-4818-b565-094d2d22afc5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:48.630218 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab401244-aa05-4818-b565-094d2d22afc5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:48.630640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab401244-aa05-4818-b565-094d2d22afc5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:48.704663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ab401244-aa05-4818-b565-094d2d22afc5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:48.714057 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ab401244-aa05-4818-b565-094d2d22afc5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.098143 Aug 07 01:25:48.714670 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 821/1482] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:48 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:49.735897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-40fe386f-3113-47b7-8347-570bc05246ea tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:49.736695 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-40fe386f-3113-47b7-8347-570bc05246ea tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:49.744768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40fe386f-3113-47b7-8347-570bc05246ea tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:49.745326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40fe386f-3113-47b7-8347-570bc05246ea tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:49.746499 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40fe386f-3113-47b7-8347-570bc05246ea tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:49.746722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40fe386f-3113-47b7-8347-570bc05246ea tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:49.747402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40fe386f-3113-47b7-8347-570bc05246ea tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:49.747604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-40fe386f-3113-47b7-8347-570bc05246ea tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:49.831458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-40fe386f-3113-47b7-8347-570bc05246ea tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:49.839995 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-40fe386f-3113-47b7-8347-570bc05246ea tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.108365 Aug 07 01:25:49.840855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 822/1483] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:49 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:50.863053 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-729135ed-a6cb-4c9d-9da7-7488dc18693b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:50.864111 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-729135ed-a6cb-4c9d-9da7-7488dc18693b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:50.871375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-729135ed-a6cb-4c9d-9da7-7488dc18693b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:50.871872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-729135ed-a6cb-4c9d-9da7-7488dc18693b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:50.872350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-729135ed-a6cb-4c9d-9da7-7488dc18693b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:50.873139 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-729135ed-a6cb-4c9d-9da7-7488dc18693b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:50.873597 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-729135ed-a6cb-4c9d-9da7-7488dc18693b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:50.874028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-729135ed-a6cb-4c9d-9da7-7488dc18693b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:50.942711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-729135ed-a6cb-4c9d-9da7-7488dc18693b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:50.952345 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-729135ed-a6cb-4c9d-9da7-7488dc18693b tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.093123 Aug 07 01:25:50.952926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 823/1484] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:50 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:51.975770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fae9d857-a565-4728-ab11-bf2a31a0d0f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:51.976811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fae9d857-a565-4728-ab11-bf2a31a0d0f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:51.985322 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fae9d857-a565-4728-ab11-bf2a31a0d0f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:51.985832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fae9d857-a565-4728-ab11-bf2a31a0d0f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:51.986439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fae9d857-a565-4728-ab11-bf2a31a0d0f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:51.987353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fae9d857-a565-4728-ab11-bf2a31a0d0f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:51.987926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fae9d857-a565-4728-ab11-bf2a31a0d0f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:51.988393 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fae9d857-a565-4728-ab11-bf2a31a0d0f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:52.067678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-fae9d857-a565-4728-ab11-bf2a31a0d0f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:52.076760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fae9d857-a565-4728-ab11-bf2a31a0d0f5 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.105540 Aug 07 01:25:52.077363 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 824/1485] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:51 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:53.099454 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e9bbff90-c3e8-4a2a-83c7-82cf26315955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:53.100261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e9bbff90-c3e8-4a2a-83c7-82cf26315955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:53.108926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9bbff90-c3e8-4a2a-83c7-82cf26315955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:53.109418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9bbff90-c3e8-4a2a-83c7-82cf26315955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:53.109965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9bbff90-c3e8-4a2a-83c7-82cf26315955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:53.110796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9bbff90-c3e8-4a2a-83c7-82cf26315955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:53.111269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9bbff90-c3e8-4a2a-83c7-82cf26315955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:53.111694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9bbff90-c3e8-4a2a-83c7-82cf26315955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:53.183133 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e9bbff90-c3e8-4a2a-83c7-82cf26315955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:53.192016 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e9bbff90-c3e8-4a2a-83c7-82cf26315955 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.096674 Aug 07 01:25:53.192743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 825/1486] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:53 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:54.214800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5545326d-9ec7-42e8-9a6b-0bd244657235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:54.215605 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5545326d-9ec7-42e8-9a6b-0bd244657235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:54.224094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5545326d-9ec7-42e8-9a6b-0bd244657235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:54.224869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5545326d-9ec7-42e8-9a6b-0bd244657235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:54.225459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5545326d-9ec7-42e8-9a6b-0bd244657235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:54.226293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5545326d-9ec7-42e8-9a6b-0bd244657235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:54.226832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5545326d-9ec7-42e8-9a6b-0bd244657235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:54.227327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5545326d-9ec7-42e8-9a6b-0bd244657235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:54.300552 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5545326d-9ec7-42e8-9a6b-0bd244657235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:54.312045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5545326d-9ec7-42e8-9a6b-0bd244657235 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.100543 Aug 07 01:25:54.312045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 826/1487] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:54 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:55.333861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-67c63d50-5861-41fe-ad83-fc65ac3f1156 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:55.334723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-67c63d50-5861-41fe-ad83-fc65ac3f1156 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:55.341460 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67c63d50-5861-41fe-ad83-fc65ac3f1156 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:55.342043 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67c63d50-5861-41fe-ad83-fc65ac3f1156 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:55.342954 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67c63d50-5861-41fe-ad83-fc65ac3f1156 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:55.343329 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67c63d50-5861-41fe-ad83-fc65ac3f1156 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:55.343858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67c63d50-5861-41fe-ad83-fc65ac3f1156 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:55.344437 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67c63d50-5861-41fe-ad83-fc65ac3f1156 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:55.349372 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:25:55.418506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-67c63d50-5861-41fe-ad83-fc65ac3f1156 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:55.429522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-67c63d50-5861-41fe-ad83-fc65ac3f1156 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.099271 Aug 07 01:25:55.430496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 827/1488] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:55 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:56.462800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-36f08d8d-2d7c-4650-b258-618975557b71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:56.463963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-36f08d8d-2d7c-4650-b258-618975557b71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:56.471757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36f08d8d-2d7c-4650-b258-618975557b71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:56.472974 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36f08d8d-2d7c-4650-b258-618975557b71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:56.472974 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36f08d8d-2d7c-4650-b258-618975557b71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:56.473869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36f08d8d-2d7c-4650-b258-618975557b71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:56.474389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36f08d8d-2d7c-4650-b258-618975557b71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:56.474882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36f08d8d-2d7c-4650-b258-618975557b71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:56.563149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-36f08d8d-2d7c-4650-b258-618975557b71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:56.577396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-36f08d8d-2d7c-4650-b258-618975557b71 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.118386 Aug 07 01:25:56.578310 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 828/1489] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:56 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:57.612305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-137b2dab-6368-419e-bf07-6f2814fe1b0f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:57.613197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-137b2dab-6368-419e-bf07-6f2814fe1b0f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:57.620344 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-137b2dab-6368-419e-bf07-6f2814fe1b0f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:57.620865 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-137b2dab-6368-419e-bf07-6f2814fe1b0f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:57.621271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-137b2dab-6368-419e-bf07-6f2814fe1b0f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:57.621986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-137b2dab-6368-419e-bf07-6f2814fe1b0f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:57.622440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-137b2dab-6368-419e-bf07-6f2814fe1b0f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:57.622869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-137b2dab-6368-419e-bf07-6f2814fe1b0f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:57.693849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-137b2dab-6368-419e-bf07-6f2814fe1b0f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:57.702683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-137b2dab-6368-419e-bf07-6f2814fe1b0f tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.094581 Aug 07 01:25:57.703398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 829/1490] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:57 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:58.724980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d9832001-17a1-4c90-9c2c-b50312d80e67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:58.725720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d9832001-17a1-4c90-9c2c-b50312d80e67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:58.733059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9832001-17a1-4c90-9c2c-b50312d80e67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:58.733825 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9832001-17a1-4c90-9c2c-b50312d80e67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:58.734387 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9832001-17a1-4c90-9c2c-b50312d80e67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:58.735262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9832001-17a1-4c90-9c2c-b50312d80e67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:58.735819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9832001-17a1-4c90-9c2c-b50312d80e67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:58.736393 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d9832001-17a1-4c90-9c2c-b50312d80e67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:58.806524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d9832001-17a1-4c90-9c2c-b50312d80e67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:58.814704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d9832001-17a1-4c90-9c2c-b50312d80e67 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.093998 Aug 07 01:25:58.815302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 830/1491] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:58 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:25:59.837602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-94b2c8d4-6b6c-496d-bb16-f8940680ac83 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:25:59.838714 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-94b2c8d4-6b6c-496d-bb16-f8940680ac83 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:25:59.847447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94b2c8d4-6b6c-496d-bb16-f8940680ac83 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:59.848113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94b2c8d4-6b6c-496d-bb16-f8940680ac83 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:59.848730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94b2c8d4-6b6c-496d-bb16-f8940680ac83 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:59.849606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94b2c8d4-6b6c-496d-bb16-f8940680ac83 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:25:59.850227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94b2c8d4-6b6c-496d-bb16-f8940680ac83 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:25:59.850762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94b2c8d4-6b6c-496d-bb16-f8940680ac83 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:25:59.929026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-94b2c8d4-6b6c-496d-bb16-f8940680ac83 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:25:59.945911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-94b2c8d4-6b6c-496d-bb16-f8940680ac83 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.112474 Aug 07 01:25:59.947596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 831/1492] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:25:59 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:26:00.982934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4e8fce21-38e2-46ba-b436-4775f06bdbf2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:26:00.983991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4e8fce21-38e2-46ba-b436-4775f06bdbf2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:26:00.991834 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e8fce21-38e2-46ba-b436-4775f06bdbf2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:26:00.992357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e8fce21-38e2-46ba-b436-4775f06bdbf2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:26:00.992804 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e8fce21-38e2-46ba-b436-4775f06bdbf2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:26:00.994162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e8fce21-38e2-46ba-b436-4775f06bdbf2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:26:00.994631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e8fce21-38e2-46ba-b436-4775f06bdbf2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:26:00.995148 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e8fce21-38e2-46ba-b436-4775f06bdbf2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:26:01.069579 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4e8fce21-38e2-46ba-b436-4775f06bdbf2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:26:01.079098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4e8fce21-38e2-46ba-b436-4775f06bdbf2 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.101074 Aug 07 01:26:01.079852 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 832/1493] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:26:00 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:26:02.101795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-53b8707a-710f-43f5-8707-92c7efd70ea6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:26:02.102926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-53b8707a-710f-43f5-8707-92c7efd70ea6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:26:02.112459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53b8707a-710f-43f5-8707-92c7efd70ea6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:26:02.113013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53b8707a-710f-43f5-8707-92c7efd70ea6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:26:02.113536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53b8707a-710f-43f5-8707-92c7efd70ea6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:26:02.115488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53b8707a-710f-43f5-8707-92c7efd70ea6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:26:02.115488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53b8707a-710f-43f5-8707-92c7efd70ea6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:26:02.115872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53b8707a-710f-43f5-8707-92c7efd70ea6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:26:02.192227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-53b8707a-710f-43f5-8707-92c7efd70ea6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:26:02.204632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-53b8707a-710f-43f5-8707-92c7efd70ea6 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.107504 Aug 07 01:26:02.205248 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 833/1494] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:26:02 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:26:03.226441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5d8526b1-700e-4c35-a4ff-0441dabb7464 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:26:03.228049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5d8526b1-700e-4c35-a4ff-0441dabb7464 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:26:03.234562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d8526b1-700e-4c35-a4ff-0441dabb7464 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:26:03.235199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d8526b1-700e-4c35-a4ff-0441dabb7464 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:26:03.235743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d8526b1-700e-4c35-a4ff-0441dabb7464 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:26:03.236628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d8526b1-700e-4c35-a4ff-0441dabb7464 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:26:03.237202 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d8526b1-700e-4c35-a4ff-0441dabb7464 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:26:03.237726 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d8526b1-700e-4c35-a4ff-0441dabb7464 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:26:03.312338 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5d8526b1-700e-4c35-a4ff-0441dabb7464 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:26:03.326194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5d8526b1-700e-4c35-a4ff-0441dabb7464 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.103417 Aug 07 01:26:03.327297 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 834/1495] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:26:03 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:26:04.359853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fde878b2-042a-40d9-8a73-b1ab94a8a793 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:26:04.360687 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fde878b2-042a-40d9-8a73-b1ab94a8a793 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:26:04.367916 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fde878b2-042a-40d9-8a73-b1ab94a8a793 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:26:04.368952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fde878b2-042a-40d9-8a73-b1ab94a8a793 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:26:04.369403 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fde878b2-042a-40d9-8a73-b1ab94a8a793 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:26:04.370142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fde878b2-042a-40d9-8a73-b1ab94a8a793 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:26:04.370602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fde878b2-042a-40d9-8a73-b1ab94a8a793 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:26:04.371077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fde878b2-042a-40d9-8a73-b1ab94a8a793 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:26:04.444706 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-fde878b2-042a-40d9-8a73-b1ab94a8a793 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:26:04.454206 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fde878b2-042a-40d9-8a73-b1ab94a8a793 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1225 microversion: 2.1 time: 0.098760 Aug 07 01:26:04.454803 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 835/1496] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:26:04 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:26:04.461501 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-77d9e6b2-9a57-49c6-b4ce-2dae56e48137 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:26:04.462420 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-77d9e6b2-9a57-49c6-b4ce-2dae56e48137 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:26:04.470894 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77d9e6b2-9a57-49c6-b4ce-2dae56e48137 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:26:04.471475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77d9e6b2-9a57-49c6-b4ce-2dae56e48137 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:26:04.471942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77d9e6b2-9a57-49c6-b4ce-2dae56e48137 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:26:04.472893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77d9e6b2-9a57-49c6-b4ce-2dae56e48137 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:26:04.473368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77d9e6b2-9a57-49c6-b4ce-2dae56e48137 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:26:04.473806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77d9e6b2-9a57-49c6-b4ce-2dae56e48137 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:26:14.468049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=37 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:26:14.495900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=56 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:26:48.007300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:26:48.007967 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:26:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518 (ip 10.222.0.26) !!! Aug 07 01:27:05.372551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-19810c6d-4632-48de-b6f4-6b2df70259e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-973f0d9b-8836-4ba6-b342-72b2da454753) Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-19810c6d-4632-48de-b6f4-6b2df70259e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Request method failure captured: Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518 HTTP/1.1 Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xGwdVHZVWnHIzGDSpTVNiz-q980RAQtQDxjuRjGuqs_HJoXOXoBhZ0fF1bbTTjk8m9Pvp2IcKkTppafVNQBMvnpaEQe9WwCTTgrFUCyEMZcpdieiPEfKYhmeOU5srzGG4O4YeIgb7xO3BdRG6OZeDumpyuz3KQXJRzGLfuf1xxtaZM Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: f7b08cc6d4a34961bdf464b58946a02e Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:27:05.377302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:27:05.378991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:27:05.378991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:27:05.378991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: f7b08cc6d4a34961bdf464b58946a02e Aug 07 01:27:05.378991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:27:05.378991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-VolumesSnapshotTestJSON-2086260047-project Aug 07 01:27:05.378991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:27:05.378991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:27:05.378991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: 6dd0ed2144164190b380ed58b3dd7cd4 Aug 07 01:27:05.378991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-VolumesSnapshotTestJSON-2086260047-project Aug 07 01:27:05.378991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-973f0d9b-8836-4ba6-b342-72b2da454753) Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'server_id': 'f481f4cc-2803-4f12-9cd2-db6055d1488c', 'id': '830234fe-bbd1-4781-97ce-8e340a8d9518'} Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 07 01:27:05.379816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-973f0d9b-8836-4ba6-b342-72b2da454753) Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in _detach_volume Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:27:05.381534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-973f0d9b-8836-4ba6-b342-72b2da454753) Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:05.383792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5152, in detach_volume Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5132, in _detach_volume Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-973f0d9b-8836-4ba6-b342-72b2da454753) Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:27:05.385815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-973f0d9b-8836-4ba6-b342-72b2da454753) Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-973f0d9b-8836-4ba6-b342-72b2da454753) Aug 07 01:27:05.388572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:05.390527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-19810c6d-4632-48de-b6f4-6b2df70259e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-973f0d9b-8836-4ba6-b342-72b2da454753) {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:27:05.390527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-19810c6d-4632-48de-b6f4-6b2df70259e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518" status: 400 len: 269 microversion: 2.1 time: 120.692957 Aug 07 01:27:05.390527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 662/1497] 10.222.0.26 () {58 vars in 1416 bytes} [Sun Aug 7 01:25:04 2022] DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518 => generated 269 bytes in 120695 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 07 01:27:05.391133 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3921b588-e2b8-4805-af8d-4cfe8bbf4e70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:27:05.392312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3921b588-e2b8-4805-af8d-4cfe8bbf4e70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:27:05.404197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3921b588-e2b8-4805-af8d-4cfe8bbf4e70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:27:05.405377 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3921b588-e2b8-4805-af8d-4cfe8bbf4e70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:27:05.406166 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3921b588-e2b8-4805-af8d-4cfe8bbf4e70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:27:05.408779 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3921b588-e2b8-4805-af8d-4cfe8bbf4e70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:27:05.408779 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3921b588-e2b8-4805-af8d-4cfe8bbf4e70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:27:05.409489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3921b588-e2b8-4805-af8d-4cfe8bbf4e70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:27:05.470303 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3921b588-e2b8-4805-af8d-4cfe8bbf4e70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:27:05.471516 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3921b588-e2b8-4805-af8d-4cfe8bbf4e70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:27:05.472311 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3921b588-e2b8-4805-af8d-4cfe8bbf4e70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:27:05.473078 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.objects.instance [None req-3921b588-e2b8-4805-af8d-4cfe8bbf4e70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lazy-loading 'fault' on Instance uuid 9968bbaa-9908-4063-bc7f-e6e805385fa2 {{(pid=117984) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:27:05.543555 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3921b588-e2b8-4805-af8d-4cfe8bbf4e70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '642126380bcf4446bf1e320f93139034', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b73b98c606a34d1faf0a31a9f6c59e56', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:27:05.555643 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3921b588-e2b8-4805-af8d-4cfe8bbf4e70 tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 200 len: 1230 microversion: 2.1 time: 0.172767 Aug 07 01:27:05.556877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 663/1498] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:27:05 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 1230 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:27:05.565476 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-26d8fc31-efc8-4cd4-a309-4a8419ce086d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:27:05.566313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-26d8fc31-efc8-4cd4-a309-4a8419ce086d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:27:05.576969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26d8fc31-efc8-4cd4-a309-4a8419ce086d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:27:05.577665 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26d8fc31-efc8-4cd4-a309-4a8419ce086d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:27:05.578427 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26d8fc31-efc8-4cd4-a309-4a8419ce086d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:27:05.579345 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26d8fc31-efc8-4cd4-a309-4a8419ce086d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:27:05.579998 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26d8fc31-efc8-4cd4-a309-4a8419ce086d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:27:05.580934 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-26d8fc31-efc8-4cd4-a309-4a8419ce086d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:27:05.969151 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.objects.instance [None req-77d9e6b2-9a57-49c6-b4ce-2dae56e48137 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lazy-loading 'services' on Instance uuid f481f4cc-2803-4f12-9cd2-db6055d1488c {{(pid=117983) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:27:06.244210 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.objects.instance [None req-26d8fc31-efc8-4cd4-a309-4a8419ce086d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lazy-loading 'services' on Instance uuid f481f4cc-2803-4f12-9cd2-db6055d1488c {{(pid=117984) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:27:06.284157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-77d9e6b2-9a57-49c6-b4ce-2dae56e48137 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82301da9-5a59-4f84-83f8-2b22b747a098) Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-77d9e6b2-9a57-49c6-b4ce-2dae56e48137 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Request method failure captured: Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518 HTTP/1.1 Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xGwdVHZVWnHIzGDSpTVNiz-q980RAQtQDxjuRjGuqs_HJoXOXoBhZ0fF1bbTTjk8m9Pvp2IcKkTppafVNQBMvnpaEQe9WwCTTgrFUCyEMZcpdieiPEfKYhmeOU5srzGG4O4YeIgb7xO3BdRG6OZeDumpyuz3KQXJRzGLfuf1xxtaZM Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: f7b08cc6d4a34961bdf464b58946a02e Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:27:06.288042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:27:06.290626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:27:06.290626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:27:06.290626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: f7b08cc6d4a34961bdf464b58946a02e Aug 07 01:27:06.290626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:27:06.290626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-VolumesSnapshotTestJSON-2086260047-project Aug 07 01:27:06.290626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:27:06.290626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:27:06.290626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 6dd0ed2144164190b380ed58b3dd7cd4 Aug 07 01:27:06.290626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-VolumesSnapshotTestJSON-2086260047-project Aug 07 01:27:06.290626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82301da9-5a59-4f84-83f8-2b22b747a098) Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'server_id': 'f481f4cc-2803-4f12-9cd2-db6055d1488c', 'id': '830234fe-bbd1-4781-97ce-8e340a8d9518'} Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 07 01:27:06.291871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82301da9-5a59-4f84-83f8-2b22b747a098) Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in _detach_volume Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:27:06.295167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82301da9-5a59-4f84-83f8-2b22b747a098) Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:06.299451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5152, in detach_volume Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5132, in _detach_volume Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82301da9-5a59-4f84-83f8-2b22b747a098) Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:27:06.303522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82301da9-5a59-4f84-83f8-2b22b747a098) Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82301da9-5a59-4f84-83f8-2b22b747a098) Aug 07 01:27:06.307856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.311287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-77d9e6b2-9a57-49c6-b4ce-2dae56e48137 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-82301da9-5a59-4f84-83f8-2b22b747a098) {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:27:06.311287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-77d9e6b2-9a57-49c6-b4ce-2dae56e48137 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518" status: 400 len: 269 microversion: 2.1 time: 61.834697 Aug 07 01:27:06.311287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 836/1499] 10.222.0.26 () {58 vars in 1416 bytes} [Sun Aug 7 01:26:04 2022] DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518 => generated 269 bytes in 61837 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 07 01:27:06.311287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fcbcd3e6-7dc6-472a-a6bf-748ad3514d4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:27:06.311923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fcbcd3e6-7dc6-472a-a6bf-748ad3514d4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [instance: 9968bbaa-9908-4063-bc7f-e6e805385fa2] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:27:06.314619 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcbcd3e6-7dc6-472a-a6bf-748ad3514d4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:27:06.315242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcbcd3e6-7dc6-472a-a6bf-748ad3514d4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:27:06.315951 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcbcd3e6-7dc6-472a-a6bf-748ad3514d4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:27:06.317041 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcbcd3e6-7dc6-472a-a6bf-748ad3514d4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:27:06.317562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcbcd3e6-7dc6-472a-a6bf-748ad3514d4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:27:06.318029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcbcd3e6-7dc6-472a-a6bf-748ad3514d4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:27:06.335810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-fcbcd3e6-7dc6-472a-a6bf-748ad3514d4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] HTTP exception thrown: Instance 9968bbaa-9908-4063-bc7f-e6e805385fa2 could not be found. Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fcbcd3e6-7dc6-472a-a6bf-748ad3514d4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Request method failure captured: Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 HTTP/1.1 Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xG-brGYSFZI7G17JATXYv2Uir-CAxcQI2npilA4n4pbbSJfYHGo0ZzCLIP1wiOOMWwkuPno6gSSoLjD_ppbsGvgBdx6cJ47Jg9xbjNA4sy9SJgr58GVoNKpAPaSgfDR_QmYnr-V975zHx1qTr0UZLUHfS9LwUSIlTDSlvpyd_GycGY Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: b73b98c606a34d1faf0a31a9f6c59e56 Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-TestVolumeBootPattern-1662403440 Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:27:06.337476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:27:06.340743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b73b98c606a34d1faf0a31a9f6c59e56"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b73b98c606a34d1faf0a31a9f6c59e56"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b73b98c606a34d1faf0a31a9f6c59e56"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b73b98c606a34d1faf0a31a9f6c59e56"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:27:06.340743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-TestVolumeBootPattern-1662403440 Aug 07 01:27:06.340743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: b73b98c606a34d1faf0a31a9f6c59e56 Aug 07 01:27:06.340743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-TestVolumeBootPattern-1662403440 Aug 07 01:27:06.340743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-TestVolumeBootPattern-1662403440-project Aug 07 01:27:06.340743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:27:06.340743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:27:06.340743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 642126380bcf4446bf1e320f93139034 Aug 07 01:27:06.340743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-TestVolumeBootPattern-1662403440-project Aug 07 01:27:06.340743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:27:06.340743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 9968bbaa-9908-4063-bc7f-e6e805385fa2 could not be found. Aug 07 01:27:06.340743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '9968bbaa-9908-4063-bc7f-e6e805385fa2'} Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9968bbaa-9908-4063-bc7f-e6e805385fa2 could not be found. Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:27:06.342625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9968bbaa-9908-4063-bc7f-e6e805385fa2 could not be found. Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9968bbaa-9908-4063-bc7f-e6e805385fa2 could not be found. Aug 07 01:27:06.345569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.348554 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fcbcd3e6-7dc6-472a-a6bf-748ad3514d4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Returning 404 to user: Instance 9968bbaa-9908-4063-bc7f-e6e805385fa2 could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:27:06.348554 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fcbcd3e6-7dc6-472a-a6bf-748ad3514d4a tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2" status: 404 len: 111 microversion: 2.1 time: 0.045508 Aug 07 01:27:06.348896 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-26d8fc31-efc8-4cd4-a309-4a8419ce086d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a0a6bc8-b6c3-4d40-ad36-b8e594f42945) Aug 07 01:27:06.349065 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 837/1500] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:27:06 2022] GET /compute/v2.1/servers/9968bbaa-9908-4063-bc7f-e6e805385fa2 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-26d8fc31-efc8-4cd4-a309-4a8419ce086d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Request method failure captured: Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518 HTTP/1.1 Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xGwdVHZVWnHIzGDSpTVNiz-q980RAQtQDxjuRjGuqs_HJoXOXoBhZ0fF1bbTTjk8m9Pvp2IcKkTppafVNQBMvnpaEQe9WwCTTgrFUCyEMZcpdieiPEfKYhmeOU5srzGG4O4YeIgb7xO3BdRG6OZeDumpyuz3KQXJRzGLfuf1xxtaZM Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: f7b08cc6d4a34961bdf464b58946a02e Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:27:06.349248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:27:06.352782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f7b08cc6d4a34961bdf464b58946a02e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:27:06.352782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:27:06.352782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: f7b08cc6d4a34961bdf464b58946a02e Aug 07 01:27:06.352782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-2086260047 Aug 07 01:27:06.352782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-VolumesSnapshotTestJSON-2086260047-project Aug 07 01:27:06.352782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:27:06.352782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:27:06.352782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: 6dd0ed2144164190b380ed58b3dd7cd4 Aug 07 01:27:06.352782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-VolumesSnapshotTestJSON-2086260047-project Aug 07 01:27:06.352782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a0a6bc8-b6c3-4d40-ad36-b8e594f42945) Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'server_id': 'f481f4cc-2803-4f12-9cd2-db6055d1488c', 'id': '830234fe-bbd1-4781-97ce-8e340a8d9518'} Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 07 01:27:06.354798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a0a6bc8-b6c3-4d40-ad36-b8e594f42945) Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in _detach_volume Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:27:06.357083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a0a6bc8-b6c3-4d40-ad36-b8e594f42945) Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:06.359468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5152, in detach_volume Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5132, in _detach_volume Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a0a6bc8-b6c3-4d40-ad36-b8e594f42945) Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:27:06.361678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a0a6bc8-b6c3-4d40-ad36-b8e594f42945) Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a0a6bc8-b6c3-4d40-ad36-b8e594f42945) Aug 07 01:27:06.364054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:27:06.366370 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-26d8fc31-efc8-4cd4-a309-4a8419ce086d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3a0a6bc8-b6c3-4d40-ad36-b8e594f42945) {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:27:06.366370 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-26d8fc31-efc8-4cd4-a309-4a8419ce086d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518" status: 400 len: 269 microversion: 2.1 time: 0.790164 Aug 07 01:27:06.366370 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 664/1501] 10.222.0.26 () {58 vars in 1416 bytes} [Sun Aug 7 01:27:05 2022] DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c/os-volume_attachments/830234fe-bbd1-4781-97ce-8e340a8d9518 => generated 269 bytes in 799 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 07 01:27:08.412028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:27:08.412797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:27:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:27:26.945249 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:27:26.946220 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:27:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:28:34.208546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:28:34.209340 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:28:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:28:44.508496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:28:44.509514 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:28:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:29:08.010207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:29:08.011113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:29:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:29:17.496888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR oslo.messaging._drivers.impl_rabbit [-] [43d5fcde-35b3-4da6-9a89-8b1380fb4929] AMQP server on 10.222.0.26:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Aug 07 01:29:17.498279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:29:17.499499 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=6 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:29:17.507599 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-079a01f3-fc51-4a15-825b-08876b6a2d38 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:17.508504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-079a01f3-fc51-4a15-825b-08876b6a2d38 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:17.523046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-079a01f3-fc51-4a15-825b-08876b6a2d38 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:17.523963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-079a01f3-fc51-4a15-825b-08876b6a2d38 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:17.524793 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-079a01f3-fc51-4a15-825b-08876b6a2d38 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:17.526073 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-079a01f3-fc51-4a15-825b-08876b6a2d38 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:17.526940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-079a01f3-fc51-4a15-825b-08876b6a2d38 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:17.528028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-079a01f3-fc51-4a15-825b-08876b6a2d38 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:17.554810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-079a01f3-fc51-4a15-825b-08876b6a2d38 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] HTTP exception thrown: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-079a01f3-fc51-4a15-825b-08876b6a2d38 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Request method failure captured: Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: DELETE /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a HTTP/1.1 Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xDuZbhbLDbrOLrZVY9vlm_HtWVxIubBKHAStAP1FnjiA_U5zjLpQJv9uy1CLGRNLdulE-dSDkxwztkKA7ofKoolemIQ0OeMpjkkOauZx4Jr-U23jKoErXsQ2oRdJqqP6y71FTJ9znyT4ExNnLWW_EBEi4UJWyaqL-JEhBPxpiLjkk0 Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Openstack-Nova-Api-Version: 2.20 Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 1b1d33b8c71e41e6bf23d4ad8590d2c2 Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:29:17.558165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:29:17.560023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:29:17.560023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:29:17.560023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 1b1d33b8c71e41e6bf23d4ad8590d2c2 Aug 07 01:29:17.560023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:29:17.560023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-AttachVolumeShelveTestJSON-734115863-project Aug 07 01:29:17.560023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:29:17.560023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:29:17.560023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 5fafe42957914b929f9ffee53ab6e7ac Aug 07 01:29:17.560023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-AttachVolumeShelveTestJSON-734115863-project Aug 07 01:29:17.560023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:29:17.560023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:29:17.560023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '00b4795f-ad25-4565-bc64-10cbae94641a'} Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:29:17.560896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:29:17.562525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.564170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-079a01f3-fc51-4a15-825b-08876b6a2d38 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Returning 404 to user: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:29:17.564170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-079a01f3-fc51-4a15-825b-08876b6a2d38 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "DELETE /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 404 len: 111 microversion: 2.20 time: 0.067191 Aug 07 01:29:17.564170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 838/1502] 10.222.0.26 () {60 vars in 1280 bytes} [Sun Aug 7 01:29:17 2022] DELETE /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 07 01:29:17.569106 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR oslo.messaging._drivers.impl_rabbit [-] [503e4d92-a7e2-4c7e-a4e9-6cdd3fd4ae0d] AMQP server on 10.222.0.26:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Aug 07 01:29:17.570104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=6 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:29:17.571412 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=16 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:29:17.577474 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-acd914a3-f7fa-4159-8093-02877dd925b1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:17.578230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-acd914a3-f7fa-4159-8093-02877dd925b1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 00b4795f-ad25-4565-bc64-10cbae94641a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:17.586943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acd914a3-f7fa-4159-8093-02877dd925b1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:17.587514 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acd914a3-f7fa-4159-8093-02877dd925b1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:17.588169 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acd914a3-f7fa-4159-8093-02877dd925b1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:17.589326 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acd914a3-f7fa-4159-8093-02877dd925b1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:17.589825 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acd914a3-f7fa-4159-8093-02877dd925b1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:17.590284 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acd914a3-f7fa-4159-8093-02877dd925b1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:17.604041 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7fc496a6-b38d-49aa-94d7-f3bfac7f9b72 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:17.604830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7fc496a6-b38d-49aa-94d7-f3bfac7f9b72 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:17.606334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-acd914a3-f7fa-4159-8093-02877dd925b1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] HTTP exception thrown: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-acd914a3-f7fa-4159-8093-02877dd925b1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Request method failure captured: Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a HTTP/1.1 Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xDuZbhbLDbrOLrZVY9vlm_HtWVxIubBKHAStAP1FnjiA_U5zjLpQJv9uy1CLGRNLdulE-dSDkxwztkKA7ofKoolemIQ0OeMpjkkOauZx4Jr-U23jKoErXsQ2oRdJqqP6y71FTJ9znyT4ExNnLWW_EBEi4UJWyaqL-JEhBPxpiLjkk0 Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Openstack-Nova-Api-Version: 2.20 Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 1b1d33b8c71e41e6bf23d4ad8590d2c2 Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:29:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:29:17.610889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:29:17.610889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:29:17.610889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 1b1d33b8c71e41e6bf23d4ad8590d2c2 Aug 07 01:29:17.610889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:29:17.610889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-AttachVolumeShelveTestJSON-734115863-project Aug 07 01:29:17.610889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:29:17.610889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:29:17.610889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: 5fafe42957914b929f9ffee53ab6e7ac Aug 07 01:29:17.610889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-AttachVolumeShelveTestJSON-734115863-project Aug 07 01:29:17.610889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:29:17.610889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:29:17.610889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '00b4795f-ad25-4565-bc64-10cbae94641a'} Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:29:17.612316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. Aug 07 01:29:17.614798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.617270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-acd914a3-f7fa-4159-8093-02877dd925b1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Returning 404 to user: Instance 00b4795f-ad25-4565-bc64-10cbae94641a could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:29:17.617270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-acd914a3-f7fa-4159-8093-02877dd925b1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a" status: 404 len: 111 microversion: 2.20 time: 0.042644 Aug 07 01:29:17.617270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 665/1503] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:29:17 2022] GET /compute/v2.1/servers/00b4795f-ad25-4565-bc64-10cbae94641a => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 07 01:29:17.617735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7fc496a6-b38d-49aa-94d7-f3bfac7f9b72 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:17.617735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7fc496a6-b38d-49aa-94d7-f3bfac7f9b72 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:17.617735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7fc496a6-b38d-49aa-94d7-f3bfac7f9b72 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:17.618204 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7fc496a6-b38d-49aa-94d7-f3bfac7f9b72 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:17.618896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7fc496a6-b38d-49aa-94d7-f3bfac7f9b72 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:17.619691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7fc496a6-b38d-49aa-94d7-f3bfac7f9b72 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:17.643916 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-7fc496a6-b38d-49aa-94d7-f3bfac7f9b72 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] HTTP exception thrown: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7fc496a6-b38d-49aa-94d7-f3bfac7f9b72 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Request method failure captured: Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: DELETE /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a HTTP/1.1 Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xD9cznP8hWf4pmxdU3hNl0ZKgrnQCZpWvAOumCUzV3uHmYQXZPbB6q2YtbZ5VRmM0fulvc4-ky89A5e1E7XmdwEvMsFQ8K0SJLQxjef3QqDDgPjeGMqBD9ErxCwISYFPrZdE0TGsrQI6E1EI8ML9K90t4_XH8zTJg5NaAH5XIw5714 Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 7fd9ab6d98a44605a57bd8ae9535a686 Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:29:17.648496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:29:17.651425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:29:17.651425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:29:17.651425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 7fd9ab6d98a44605a57bd8ae9535a686 Aug 07 01:29:17.651425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:29:17.651425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-AttachVolumeTestJSON-1659395094-project Aug 07 01:29:17.651425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:29:17.651425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:29:17.651425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 4ae9c7ef17b746fc8f87e3c22a241b76 Aug 07 01:29:17.651425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-AttachVolumeTestJSON-1659395094-project Aug 07 01:29:17.651425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:29:17.651425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:29:17.651425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '8e8c48f1-19d6-4691-9e54-e5d84962485a'} Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:29:17.653067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:29:17.655843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.658921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7fc496a6-b38d-49aa-94d7-f3bfac7f9b72 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Returning 404 to user: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:29:17.658921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7fc496a6-b38d-49aa-94d7-f3bfac7f9b72 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "DELETE /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 404 len: 111 microversion: 2.1 time: 0.052936 Aug 07 01:29:17.659321 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 839/1504] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:29:17 2022] DELETE /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:29:17.667097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9b74695f-c499-4636-a8ea-685ab75b10fb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:17.668159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9b74695f-c499-4636-a8ea-685ab75b10fb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 8e8c48f1-19d6-4691-9e54-e5d84962485a] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:17.678784 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9b74695f-c499-4636-a8ea-685ab75b10fb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:17.679457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9b74695f-c499-4636-a8ea-685ab75b10fb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:17.680270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9b74695f-c499-4636-a8ea-685ab75b10fb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:17.681645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9b74695f-c499-4636-a8ea-685ab75b10fb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:17.682331 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9b74695f-c499-4636-a8ea-685ab75b10fb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:17.682975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9b74695f-c499-4636-a8ea-685ab75b10fb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:17.699058 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-9b74695f-c499-4636-a8ea-685ab75b10fb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] HTTP exception thrown: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9b74695f-c499-4636-a8ea-685ab75b10fb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Request method failure captured: Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a HTTP/1.1 Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xD9cznP8hWf4pmxdU3hNl0ZKgrnQCZpWvAOumCUzV3uHmYQXZPbB6q2YtbZ5VRmM0fulvc4-ky89A5e1E7XmdwEvMsFQ8K0SJLQxjef3QqDDgPjeGMqBD9ErxCwISYFPrZdE0TGsrQI6E1EI8ML9K90t4_XH8zTJg5NaAH5XIw5714 Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 7fd9ab6d98a44605a57bd8ae9535a686 Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:29:17.700687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:29:17.702902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:29:17.702902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:29:17.702902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 7fd9ab6d98a44605a57bd8ae9535a686 Aug 07 01:29:17.702902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:29:17.702902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-AttachVolumeTestJSON-1659395094-project Aug 07 01:29:17.702902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:29:17.702902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:29:17.702902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: 4ae9c7ef17b746fc8f87e3c22a241b76 Aug 07 01:29:17.702902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-AttachVolumeTestJSON-1659395094-project Aug 07 01:29:17.702902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:29:17.702902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:29:17.702902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '8e8c48f1-19d6-4691-9e54-e5d84962485a'} Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:29:17.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. Aug 07 01:29:17.707692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:29:17.710126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9b74695f-c499-4636-a8ea-685ab75b10fb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Returning 404 to user: Instance 8e8c48f1-19d6-4691-9e54-e5d84962485a could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:29:17.710126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9b74695f-c499-4636-a8ea-685ab75b10fb tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a" status: 404 len: 111 microversion: 2.1 time: 0.041189 Aug 07 01:29:17.710126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 666/1505] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:17 2022] GET /compute/v2.1/servers/8e8c48f1-19d6-4691-9e54-e5d84962485a => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:29:18.060197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-acb28048-25d3-4aac-96f9-99e3dbe411f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:18.061812 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-acb28048-25d3-4aac-96f9-99e3dbe411f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:18.072041 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acb28048-25d3-4aac-96f9-99e3dbe411f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:18.072705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acb28048-25d3-4aac-96f9-99e3dbe411f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:18.073177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acb28048-25d3-4aac-96f9-99e3dbe411f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:18.074002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acb28048-25d3-4aac-96f9-99e3dbe411f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:18.074762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acb28048-25d3-4aac-96f9-99e3dbe411f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:18.075479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-acb28048-25d3-4aac-96f9-99e3dbe411f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:18.107281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-acb28048-25d3-4aac-96f9-99e3dbe411f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Going to try to terminate instance {{(pid=117983) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:29:18.208964 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-acb28048-25d3-4aac-96f9-99e3dbe411f8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "DELETE /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 204 len: 0 microversion: 2.1 time: 0.153179 Aug 07 01:29:18.209674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 840/1506] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:29:18 2022] DELETE /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 07 01:29:18.222670 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2fc6a8ae-01ff-410b-aadc-782ac8bd28af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:18.223683 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2fc6a8ae-01ff-410b-aadc-782ac8bd28af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:18.235337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2fc6a8ae-01ff-410b-aadc-782ac8bd28af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:18.235944 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2fc6a8ae-01ff-410b-aadc-782ac8bd28af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:18.236425 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2fc6a8ae-01ff-410b-aadc-782ac8bd28af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:18.237187 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2fc6a8ae-01ff-410b-aadc-782ac8bd28af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:18.237717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2fc6a8ae-01ff-410b-aadc-782ac8bd28af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:18.238757 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2fc6a8ae-01ff-410b-aadc-782ac8bd28af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:18.406587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2fc6a8ae-01ff-410b-aadc-782ac8bd28af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:18.415586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2fc6a8ae-01ff-410b-aadc-782ac8bd28af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1736 microversion: 2.1 time: 0.201064 Aug 07 01:29:18.421786 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 667/1507] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:18 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1736 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:29:19.449164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a7827408-5644-4be0-a120-ecc802631ef0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:19.450799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a7827408-5644-4be0-a120-ecc802631ef0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:19.463262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7827408-5644-4be0-a120-ecc802631ef0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:19.464398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7827408-5644-4be0-a120-ecc802631ef0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:19.465142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7827408-5644-4be0-a120-ecc802631ef0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:19.467247 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7827408-5644-4be0-a120-ecc802631ef0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:19.467247 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7827408-5644-4be0-a120-ecc802631ef0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:19.467556 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7827408-5644-4be0-a120-ecc802631ef0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:19.504528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] [43d5fcde-35b3-4da6-9a89-8b1380fb4929] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 34462. Aug 07 01:29:19.562845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a7827408-5644-4be0-a120-ecc802631ef0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:19.572813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a7827408-5644-4be0-a120-ecc802631ef0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1668 microversion: 2.1 time: 0.129468 Aug 07 01:29:19.573844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 841/1508] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:19 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1668 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:19.976948 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] [503e4d92-a7e2-4c7e-a4e9-6cdd3fd4ae0d] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 34476. Aug 07 01:29:20.133997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-b44907d9-6a9f-4d12-a3d5-393158a42b3d req-92f25339-e050-4f61-b9f9-237d256deaa9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6ac28640-d6da-4cfc-b7ad-0d69136ac1fd", "name": "network-vif-deleted", "tag": "4d7d0711-c6ef-4ae2-b272-c2cf175993d4"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:29:20.149426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-b44907d9-6a9f-4d12-a3d5-393158a42b3d req-92f25339-e050-4f61-b9f9-237d256deaa9 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:20.150023 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-b44907d9-6a9f-4d12-a3d5-393158a42b3d req-92f25339-e050-4f61-b9f9-237d256deaa9 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:20.150703 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-b44907d9-6a9f-4d12-a3d5-393158a42b3d req-92f25339-e050-4f61-b9f9-237d256deaa9 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:20.164104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-b44907d9-6a9f-4d12-a3d5-393158a42b3d req-92f25339-e050-4f61-b9f9-237d256deaa9 service nova] Creating event network-vif-deleted:4d7d0711-c6ef-4ae2-b272-c2cf175993d4 for instance 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd on hv-cinder-maste Aug 07 01:29:20.172187 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-b44907d9-6a9f-4d12-a3d5-393158a42b3d req-92f25339-e050-4f61-b9f9-237d256deaa9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.211797 Aug 07 01:29:20.172842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 668/1509] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:29:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:29:20.598683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0d80dddd-484f-4775-abfe-041e53ab1e83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:20.599569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0d80dddd-484f-4775-abfe-041e53ab1e83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:20.606840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d80dddd-484f-4775-abfe-041e53ab1e83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:20.607342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d80dddd-484f-4775-abfe-041e53ab1e83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:20.607790 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d80dddd-484f-4775-abfe-041e53ab1e83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:20.608566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d80dddd-484f-4775-abfe-041e53ab1e83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:20.609100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d80dddd-484f-4775-abfe-041e53ab1e83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:20.609469 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d80dddd-484f-4775-abfe-041e53ab1e83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:20.678186 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0d80dddd-484f-4775-abfe-041e53ab1e83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:20.688697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0d80dddd-484f-4775-abfe-041e53ab1e83 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097517 Aug 07 01:29:20.689482 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 842/1510] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:20 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:21.719131 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-583345e3-dd19-48e0-a89a-e644d8f72759 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:21.719841 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-583345e3-dd19-48e0-a89a-e644d8f72759 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:21.727414 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-583345e3-dd19-48e0-a89a-e644d8f72759 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:21.727920 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-583345e3-dd19-48e0-a89a-e644d8f72759 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:21.728394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-583345e3-dd19-48e0-a89a-e644d8f72759 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:21.729137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-583345e3-dd19-48e0-a89a-e644d8f72759 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:21.729601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-583345e3-dd19-48e0-a89a-e644d8f72759 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:21.730029 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-583345e3-dd19-48e0-a89a-e644d8f72759 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:22.106939 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-583345e3-dd19-48e0-a89a-e644d8f72759 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:22.115897 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-583345e3-dd19-48e0-a89a-e644d8f72759 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.401851 Aug 07 01:29:22.116555 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 669/1511] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:21 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:29:23.141921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-995eff27-0e67-4ef3-83a4-5f2c679c8519 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:23.143043 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-995eff27-0e67-4ef3-83a4-5f2c679c8519 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:23.155319 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-995eff27-0e67-4ef3-83a4-5f2c679c8519 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:23.156190 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-995eff27-0e67-4ef3-83a4-5f2c679c8519 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:23.157014 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-995eff27-0e67-4ef3-83a4-5f2c679c8519 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:23.158297 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-995eff27-0e67-4ef3-83a4-5f2c679c8519 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:23.159412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-995eff27-0e67-4ef3-83a4-5f2c679c8519 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:23.160374 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-995eff27-0e67-4ef3-83a4-5f2c679c8519 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:23.234038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-995eff27-0e67-4ef3-83a4-5f2c679c8519 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:23.243651 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-995eff27-0e67-4ef3-83a4-5f2c679c8519 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.109396 Aug 07 01:29:23.244275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 843/1512] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:23 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:24.279738 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-870c75b7-3e28-4902-81d3-6337a458882e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:24.280398 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-870c75b7-3e28-4902-81d3-6337a458882e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:24.290387 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-870c75b7-3e28-4902-81d3-6337a458882e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:24.290974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-870c75b7-3e28-4902-81d3-6337a458882e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:24.291491 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-870c75b7-3e28-4902-81d3-6337a458882e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:24.292415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-870c75b7-3e28-4902-81d3-6337a458882e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:24.292926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-870c75b7-3e28-4902-81d3-6337a458882e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:24.293406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-870c75b7-3e28-4902-81d3-6337a458882e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:24.365467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-870c75b7-3e28-4902-81d3-6337a458882e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:24.373715 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-870c75b7-3e28-4902-81d3-6337a458882e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098166 Aug 07 01:29:24.374299 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 670/1513] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:24 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:25.395741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ffb46790-6045-4b50-90c8-03ad754da614 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:25.396552 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ffb46790-6045-4b50-90c8-03ad754da614 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:25.403623 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ffb46790-6045-4b50-90c8-03ad754da614 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:25.404110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ffb46790-6045-4b50-90c8-03ad754da614 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:25.404570 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ffb46790-6045-4b50-90c8-03ad754da614 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:25.405324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ffb46790-6045-4b50-90c8-03ad754da614 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:25.405775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ffb46790-6045-4b50-90c8-03ad754da614 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:25.406221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ffb46790-6045-4b50-90c8-03ad754da614 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:25.493784 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ffb46790-6045-4b50-90c8-03ad754da614 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:25.502703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ffb46790-6045-4b50-90c8-03ad754da614 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.111229 Aug 07 01:29:25.503922 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 844/1514] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:25 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:26.535274 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2d101ef5-7fa3-4b31-9212-98c34631e745 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:26.535880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2d101ef5-7fa3-4b31-9212-98c34631e745 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:26.542961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2d101ef5-7fa3-4b31-9212-98c34631e745 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:26.543445 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2d101ef5-7fa3-4b31-9212-98c34631e745 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:26.543894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2d101ef5-7fa3-4b31-9212-98c34631e745 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:26.544647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2d101ef5-7fa3-4b31-9212-98c34631e745 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:26.545107 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2d101ef5-7fa3-4b31-9212-98c34631e745 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:26.545538 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2d101ef5-7fa3-4b31-9212-98c34631e745 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:26.616528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2d101ef5-7fa3-4b31-9212-98c34631e745 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:26.625056 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2d101ef5-7fa3-4b31-9212-98c34631e745 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094401 Aug 07 01:29:26.625645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 671/1515] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:26 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:27.650470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fc4dd4d8-8f38-4d0c-bb3f-4d5debcfd25a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:27.651063 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fc4dd4d8-8f38-4d0c-bb3f-4d5debcfd25a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:27.663062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc4dd4d8-8f38-4d0c-bb3f-4d5debcfd25a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:27.663928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc4dd4d8-8f38-4d0c-bb3f-4d5debcfd25a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:27.664790 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc4dd4d8-8f38-4d0c-bb3f-4d5debcfd25a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:27.666108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc4dd4d8-8f38-4d0c-bb3f-4d5debcfd25a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:27.666963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc4dd4d8-8f38-4d0c-bb3f-4d5debcfd25a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:27.667769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc4dd4d8-8f38-4d0c-bb3f-4d5debcfd25a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:27.773235 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-fc4dd4d8-8f38-4d0c-bb3f-4d5debcfd25a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:27.788915 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fc4dd4d8-8f38-4d0c-bb3f-4d5debcfd25a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.144600 Aug 07 01:29:27.789963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 845/1516] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:27 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:28.824104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7dd7389a-9164-406d-82ec-46863f355ee6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:28.824816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7dd7389a-9164-406d-82ec-46863f355ee6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:28.833316 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7dd7389a-9164-406d-82ec-46863f355ee6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:28.833797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7dd7389a-9164-406d-82ec-46863f355ee6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:28.834241 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7dd7389a-9164-406d-82ec-46863f355ee6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:28.835066 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7dd7389a-9164-406d-82ec-46863f355ee6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:28.835526 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7dd7389a-9164-406d-82ec-46863f355ee6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:28.835960 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7dd7389a-9164-406d-82ec-46863f355ee6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:28.925525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7dd7389a-9164-406d-82ec-46863f355ee6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:28.935020 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7dd7389a-9164-406d-82ec-46863f355ee6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.115483 Aug 07 01:29:28.935629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 672/1517] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:28 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:29.959566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fcb6e5e9-daca-4cc8-8466-7a1597f68b55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:29.960428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fcb6e5e9-daca-4cc8-8466-7a1597f68b55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:29.969303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcb6e5e9-daca-4cc8-8466-7a1597f68b55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:29.969801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcb6e5e9-daca-4cc8-8466-7a1597f68b55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:29.970244 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcb6e5e9-daca-4cc8-8466-7a1597f68b55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:29.970982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcb6e5e9-daca-4cc8-8466-7a1597f68b55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:29.971443 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcb6e5e9-daca-4cc8-8466-7a1597f68b55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:29.972596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fcb6e5e9-daca-4cc8-8466-7a1597f68b55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:30.046826 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-fcb6e5e9-daca-4cc8-8466-7a1597f68b55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:30.056742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fcb6e5e9-daca-4cc8-8466-7a1597f68b55 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.102430 Aug 07 01:29:30.057350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 846/1518] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:29 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:31.080309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-77234910-cf7d-4c1b-bc71-63c69597253a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:31.080931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-77234910-cf7d-4c1b-bc71-63c69597253a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:31.088881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77234910-cf7d-4c1b-bc71-63c69597253a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:31.089375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77234910-cf7d-4c1b-bc71-63c69597253a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:31.089813 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77234910-cf7d-4c1b-bc71-63c69597253a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:31.090717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77234910-cf7d-4c1b-bc71-63c69597253a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:31.091180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77234910-cf7d-4c1b-bc71-63c69597253a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:31.091661 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77234910-cf7d-4c1b-bc71-63c69597253a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:31.183812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-77234910-cf7d-4c1b-bc71-63c69597253a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:31.194369 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-77234910-cf7d-4c1b-bc71-63c69597253a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.118155 Aug 07 01:29:31.194969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 673/1519] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:31 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:32.217702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4ec1fc65-55e4-4a12-9f3e-cfba73f4f019 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:32.218336 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4ec1fc65-55e4-4a12-9f3e-cfba73f4f019 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:32.226214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4ec1fc65-55e4-4a12-9f3e-cfba73f4f019 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:32.226700 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4ec1fc65-55e4-4a12-9f3e-cfba73f4f019 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:32.227142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4ec1fc65-55e4-4a12-9f3e-cfba73f4f019 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:32.227872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4ec1fc65-55e4-4a12-9f3e-cfba73f4f019 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:32.228429 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4ec1fc65-55e4-4a12-9f3e-cfba73f4f019 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:32.228862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4ec1fc65-55e4-4a12-9f3e-cfba73f4f019 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:32.299015 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4ec1fc65-55e4-4a12-9f3e-cfba73f4f019 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:32.308829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4ec1fc65-55e4-4a12-9f3e-cfba73f4f019 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.095061 Aug 07 01:29:32.309415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 847/1520] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:32 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:33.345245 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4836fab1-e34b-46b8-8450-7e3fc24465dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:33.346121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4836fab1-e34b-46b8-8450-7e3fc24465dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:33.353036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4836fab1-e34b-46b8-8450-7e3fc24465dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:33.353545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4836fab1-e34b-46b8-8450-7e3fc24465dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:33.353989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4836fab1-e34b-46b8-8450-7e3fc24465dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:33.354817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4836fab1-e34b-46b8-8450-7e3fc24465dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:33.355640 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4836fab1-e34b-46b8-8450-7e3fc24465dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:33.356476 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4836fab1-e34b-46b8-8450-7e3fc24465dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:33.439323 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4836fab1-e34b-46b8-8450-7e3fc24465dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:33.449027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4836fab1-e34b-46b8-8450-7e3fc24465dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.107624 Aug 07 01:29:33.449629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 674/1521] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:33 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:34.470993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3087bc01-f3e8-4ebb-b9f9-8b78887fc6a6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:34.471756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3087bc01-f3e8-4ebb-b9f9-8b78887fc6a6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:34.479778 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3087bc01-f3e8-4ebb-b9f9-8b78887fc6a6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:34.480297 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3087bc01-f3e8-4ebb-b9f9-8b78887fc6a6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:34.480772 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3087bc01-f3e8-4ebb-b9f9-8b78887fc6a6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:34.481546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3087bc01-f3e8-4ebb-b9f9-8b78887fc6a6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:34.482006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3087bc01-f3e8-4ebb-b9f9-8b78887fc6a6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:34.482427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3087bc01-f3e8-4ebb-b9f9-8b78887fc6a6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:34.554816 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3087bc01-f3e8-4ebb-b9f9-8b78887fc6a6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:34.570553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3087bc01-f3e8-4ebb-b9f9-8b78887fc6a6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.103161 Aug 07 01:29:34.571606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 848/1522] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:34 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:35.593659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6a35e129-fcd4-4f35-a71a-db942ecbb47f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:35.594320 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6a35e129-fcd4-4f35-a71a-db942ecbb47f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:35.601675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6a35e129-fcd4-4f35-a71a-db942ecbb47f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:35.602166 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6a35e129-fcd4-4f35-a71a-db942ecbb47f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:35.602756 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6a35e129-fcd4-4f35-a71a-db942ecbb47f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:35.603358 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6a35e129-fcd4-4f35-a71a-db942ecbb47f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:35.603823 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6a35e129-fcd4-4f35-a71a-db942ecbb47f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:35.604284 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6a35e129-fcd4-4f35-a71a-db942ecbb47f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:35.674485 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6a35e129-fcd4-4f35-a71a-db942ecbb47f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:35.683887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6a35e129-fcd4-4f35-a71a-db942ecbb47f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094111 Aug 07 01:29:35.685203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 675/1523] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:35 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:36.722845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-79e47894-8267-4892-a26e-742d74b098ed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:36.723997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-79e47894-8267-4892-a26e-742d74b098ed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:36.731563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-79e47894-8267-4892-a26e-742d74b098ed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:36.732058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-79e47894-8267-4892-a26e-742d74b098ed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:36.732545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-79e47894-8267-4892-a26e-742d74b098ed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:36.733293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-79e47894-8267-4892-a26e-742d74b098ed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:36.733748 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-79e47894-8267-4892-a26e-742d74b098ed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:36.734183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-79e47894-8267-4892-a26e-742d74b098ed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:36.814555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-79e47894-8267-4892-a26e-742d74b098ed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:36.823778 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-79e47894-8267-4892-a26e-742d74b098ed tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.107559 Aug 07 01:29:36.824686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 849/1524] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:36 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:37.848223 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3b9d6327-22ad-40e8-85b5-6eb7d61ff81f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:37.849010 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3b9d6327-22ad-40e8-85b5-6eb7d61ff81f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:37.856521 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b9d6327-22ad-40e8-85b5-6eb7d61ff81f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:37.857011 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b9d6327-22ad-40e8-85b5-6eb7d61ff81f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:37.857448 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b9d6327-22ad-40e8-85b5-6eb7d61ff81f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:37.858256 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b9d6327-22ad-40e8-85b5-6eb7d61ff81f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:37.858705 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b9d6327-22ad-40e8-85b5-6eb7d61ff81f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:37.859123 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b9d6327-22ad-40e8-85b5-6eb7d61ff81f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:37.930432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3b9d6327-22ad-40e8-85b5-6eb7d61ff81f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:37.939354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3b9d6327-22ad-40e8-85b5-6eb7d61ff81f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.096195 Aug 07 01:29:37.940349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 676/1525] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:37 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:38.961189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ea1ef296-b027-4573-abcc-d10a517248a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:38.961928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ea1ef296-b027-4573-abcc-d10a517248a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:38.969209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ea1ef296-b027-4573-abcc-d10a517248a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:38.969701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ea1ef296-b027-4573-abcc-d10a517248a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:38.970143 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ea1ef296-b027-4573-abcc-d10a517248a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:38.970847 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ea1ef296-b027-4573-abcc-d10a517248a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:38.971294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ea1ef296-b027-4573-abcc-d10a517248a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:38.971786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ea1ef296-b027-4573-abcc-d10a517248a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:39.039746 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ea1ef296-b027-4573-abcc-d10a517248a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:39.048414 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ea1ef296-b027-4573-abcc-d10a517248a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.091015 Aug 07 01:29:39.049141 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 850/1526] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:38 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:40.074603 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7b09cd86-622c-4b11-a1a1-ae96228b8fd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:40.075708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7b09cd86-622c-4b11-a1a1-ae96228b8fd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:40.089211 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7b09cd86-622c-4b11-a1a1-ae96228b8fd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:40.090160 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7b09cd86-622c-4b11-a1a1-ae96228b8fd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:40.091140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7b09cd86-622c-4b11-a1a1-ae96228b8fd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:40.092480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7b09cd86-622c-4b11-a1a1-ae96228b8fd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:40.093277 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7b09cd86-622c-4b11-a1a1-ae96228b8fd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:40.094045 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7b09cd86-622c-4b11-a1a1-ae96228b8fd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:40.202703 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7b09cd86-622c-4b11-a1a1-ae96228b8fd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:40.212677 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7b09cd86-622c-4b11-a1a1-ae96228b8fd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.145610 Aug 07 01:29:40.213506 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 677/1527] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:40 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:41.239993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ff4be90f-5182-41d8-8cc0-01a7741ed872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:41.241095 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ff4be90f-5182-41d8-8cc0-01a7741ed872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:41.252603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ff4be90f-5182-41d8-8cc0-01a7741ed872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:41.253500 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ff4be90f-5182-41d8-8cc0-01a7741ed872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:41.254286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ff4be90f-5182-41d8-8cc0-01a7741ed872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:41.255499 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ff4be90f-5182-41d8-8cc0-01a7741ed872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:41.256341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ff4be90f-5182-41d8-8cc0-01a7741ed872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:41.257099 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ff4be90f-5182-41d8-8cc0-01a7741ed872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:41.330052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ff4be90f-5182-41d8-8cc0-01a7741ed872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:41.339360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ff4be90f-5182-41d8-8cc0-01a7741ed872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.106615 Aug 07 01:29:41.339963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 851/1528] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:41 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:42.363806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0a99bb05-09e5-41b9-9d82-aa51be61618e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:42.364538 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0a99bb05-09e5-41b9-9d82-aa51be61618e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:42.371101 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0a99bb05-09e5-41b9-9d82-aa51be61618e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:42.371597 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0a99bb05-09e5-41b9-9d82-aa51be61618e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:42.372042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0a99bb05-09e5-41b9-9d82-aa51be61618e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:42.372822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0a99bb05-09e5-41b9-9d82-aa51be61618e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:42.373297 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0a99bb05-09e5-41b9-9d82-aa51be61618e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:42.373734 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0a99bb05-09e5-41b9-9d82-aa51be61618e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:42.444121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0a99bb05-09e5-41b9-9d82-aa51be61618e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:42.453962 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0a99bb05-09e5-41b9-9d82-aa51be61618e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.095703 Aug 07 01:29:42.454576 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 678/1529] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:42 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:43.476918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0c712cb7-d836-45ce-ac92-01a776550923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:43.477955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0c712cb7-d836-45ce-ac92-01a776550923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:43.485631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c712cb7-d836-45ce-ac92-01a776550923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:43.486271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c712cb7-d836-45ce-ac92-01a776550923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:43.486823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c712cb7-d836-45ce-ac92-01a776550923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:43.488342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c712cb7-d836-45ce-ac92-01a776550923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:43.489000 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c712cb7-d836-45ce-ac92-01a776550923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:43.489705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c712cb7-d836-45ce-ac92-01a776550923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:43.563693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0c712cb7-d836-45ce-ac92-01a776550923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:43.573292 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0c712cb7-d836-45ce-ac92-01a776550923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.100534 Aug 07 01:29:43.573894 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 852/1530] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:43 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:29:44.605436 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ca5d889c-3a6b-4e98-9df9-63138bcfdef6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:44.606064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ca5d889c-3a6b-4e98-9df9-63138bcfdef6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:44.613406 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca5d889c-3a6b-4e98-9df9-63138bcfdef6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:44.613890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca5d889c-3a6b-4e98-9df9-63138bcfdef6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:44.614320 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca5d889c-3a6b-4e98-9df9-63138bcfdef6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:44.615101 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca5d889c-3a6b-4e98-9df9-63138bcfdef6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:44.615548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca5d889c-3a6b-4e98-9df9-63138bcfdef6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:44.615976 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca5d889c-3a6b-4e98-9df9-63138bcfdef6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:44.693136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ca5d889c-3a6b-4e98-9df9-63138bcfdef6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:44.703238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ca5d889c-3a6b-4e98-9df9-63138bcfdef6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.101684 Aug 07 01:29:44.704093 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 679/1531] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:44 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:45.725988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6623f117-f363-4e5d-8ba3-4373e64817ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:45.726600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6623f117-f363-4e5d-8ba3-4373e64817ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:45.733744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6623f117-f363-4e5d-8ba3-4373e64817ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:45.734315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6623f117-f363-4e5d-8ba3-4373e64817ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:45.734872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6623f117-f363-4e5d-8ba3-4373e64817ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:45.735606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6623f117-f363-4e5d-8ba3-4373e64817ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:45.736056 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6623f117-f363-4e5d-8ba3-4373e64817ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:45.736524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6623f117-f363-4e5d-8ba3-4373e64817ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:45.835305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6623f117-f363-4e5d-8ba3-4373e64817ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:45.844942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6623f117-f363-4e5d-8ba3-4373e64817ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.122737 Aug 07 01:29:45.845861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 853/1532] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:45 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:29:46.870410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-dfd0ccaa-dfe8-49f9-ba7a-ec353d0fabad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:46.871052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-dfd0ccaa-dfe8-49f9-ba7a-ec353d0fabad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:46.878531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dfd0ccaa-dfe8-49f9-ba7a-ec353d0fabad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:46.879022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dfd0ccaa-dfe8-49f9-ba7a-ec353d0fabad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:46.879466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dfd0ccaa-dfe8-49f9-ba7a-ec353d0fabad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:46.880201 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dfd0ccaa-dfe8-49f9-ba7a-ec353d0fabad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:46.880697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dfd0ccaa-dfe8-49f9-ba7a-ec353d0fabad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:46.881126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dfd0ccaa-dfe8-49f9-ba7a-ec353d0fabad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:46.961712 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dfd0ccaa-dfe8-49f9-ba7a-ec353d0fabad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:46.970417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-dfd0ccaa-dfe8-49f9-ba7a-ec353d0fabad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.103902 Aug 07 01:29:46.971010 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 680/1533] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:46 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:47.992179 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4abe7881-c21d-4d69-884c-3b525fab4231 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:47.992972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4abe7881-c21d-4d69-884c-3b525fab4231 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:47.995404 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=38,nova_api:UPDATE=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:29:48.001661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4abe7881-c21d-4d69-884c-3b525fab4231 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:48.002166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4abe7881-c21d-4d69-884c-3b525fab4231 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:48.002771 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4abe7881-c21d-4d69-884c-3b525fab4231 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:48.003573 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4abe7881-c21d-4d69-884c-3b525fab4231 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:48.004042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4abe7881-c21d-4d69-884c-3b525fab4231 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:48.005427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4abe7881-c21d-4d69-884c-3b525fab4231 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:48.009874 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:29:48.086738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4abe7881-c21d-4d69-884c-3b525fab4231 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:48.096280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4abe7881-c21d-4d69-884c-3b525fab4231 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.108002 Aug 07 01:29:48.098723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 854/1534] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:47 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:49.118771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fe514da2-0cd1-4b95-babe-38ac52fca49f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:49.119473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fe514da2-0cd1-4b95-babe-38ac52fca49f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:49.121935 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=33 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:29:49.128886 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe514da2-0cd1-4b95-babe-38ac52fca49f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:49.129385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe514da2-0cd1-4b95-babe-38ac52fca49f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:49.129830 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe514da2-0cd1-4b95-babe-38ac52fca49f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:49.130559 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe514da2-0cd1-4b95-babe-38ac52fca49f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:49.131049 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe514da2-0cd1-4b95-babe-38ac52fca49f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:49.131571 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fe514da2-0cd1-4b95-babe-38ac52fca49f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:49.135169 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=85 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:29:49.210162 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fe514da2-0cd1-4b95-babe-38ac52fca49f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:49.218667 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fe514da2-0cd1-4b95-babe-38ac52fca49f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.103967 Aug 07 01:29:49.219263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 681/1535] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:49 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:50.241808 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-522b1e09-f05a-4f65-a1b6-89d327623271 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:50.242495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-522b1e09-f05a-4f65-a1b6-89d327623271 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:50.248978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-522b1e09-f05a-4f65-a1b6-89d327623271 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:50.249470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-522b1e09-f05a-4f65-a1b6-89d327623271 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:50.249911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-522b1e09-f05a-4f65-a1b6-89d327623271 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:50.250625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-522b1e09-f05a-4f65-a1b6-89d327623271 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:50.251079 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-522b1e09-f05a-4f65-a1b6-89d327623271 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:50.251498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-522b1e09-f05a-4f65-a1b6-89d327623271 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:50.345937 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-522b1e09-f05a-4f65-a1b6-89d327623271 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:50.361318 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-522b1e09-f05a-4f65-a1b6-89d327623271 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.123771 Aug 07 01:29:50.362956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 855/1536] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:50 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:29:51.396877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-90fbef04-0f24-4187-913d-8f2ac402b9fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:51.397737 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-90fbef04-0f24-4187-913d-8f2ac402b9fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:51.404565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-90fbef04-0f24-4187-913d-8f2ac402b9fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:51.405059 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-90fbef04-0f24-4187-913d-8f2ac402b9fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:51.405659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-90fbef04-0f24-4187-913d-8f2ac402b9fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:51.406398 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-90fbef04-0f24-4187-913d-8f2ac402b9fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:51.406857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-90fbef04-0f24-4187-913d-8f2ac402b9fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:51.407295 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-90fbef04-0f24-4187-913d-8f2ac402b9fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:51.483019 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-90fbef04-0f24-4187-913d-8f2ac402b9fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:51.492908 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-90fbef04-0f24-4187-913d-8f2ac402b9fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.101094 Aug 07 01:29:51.493680 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 682/1537] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:51 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:29:52.516326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-192e8a2c-fe9b-40ef-9ad2-dfdf5195b928 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:52.517361 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-192e8a2c-fe9b-40ef-9ad2-dfdf5195b928 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:52.524189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-192e8a2c-fe9b-40ef-9ad2-dfdf5195b928 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:52.524719 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-192e8a2c-fe9b-40ef-9ad2-dfdf5195b928 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:52.525170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-192e8a2c-fe9b-40ef-9ad2-dfdf5195b928 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:52.525991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-192e8a2c-fe9b-40ef-9ad2-dfdf5195b928 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:52.526464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-192e8a2c-fe9b-40ef-9ad2-dfdf5195b928 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:52.526903 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-192e8a2c-fe9b-40ef-9ad2-dfdf5195b928 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:52.597907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-192e8a2c-fe9b-40ef-9ad2-dfdf5195b928 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:52.606769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-192e8a2c-fe9b-40ef-9ad2-dfdf5195b928 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094708 Aug 07 01:29:52.607372 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 856/1538] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:52 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:53.641113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-468c8e16-bd77-4800-ac8c-a79c97c3c4e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:53.641826 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-468c8e16-bd77-4800-ac8c-a79c97c3c4e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:53.650361 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-468c8e16-bd77-4800-ac8c-a79c97c3c4e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:53.650939 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-468c8e16-bd77-4800-ac8c-a79c97c3c4e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:53.651415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-468c8e16-bd77-4800-ac8c-a79c97c3c4e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:53.652138 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-468c8e16-bd77-4800-ac8c-a79c97c3c4e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:53.652641 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-468c8e16-bd77-4800-ac8c-a79c97c3c4e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:53.653162 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-468c8e16-bd77-4800-ac8c-a79c97c3c4e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:53.744800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-468c8e16-bd77-4800-ac8c-a79c97c3c4e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:53.753587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-468c8e16-bd77-4800-ac8c-a79c97c3c4e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.117103 Aug 07 01:29:53.754185 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 683/1539] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:53 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:29:54.789688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dedb37c6-8fad-4249-8c70-083fc8f5bbb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:54.790483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dedb37c6-8fad-4249-8c70-083fc8f5bbb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:54.797711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dedb37c6-8fad-4249-8c70-083fc8f5bbb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:54.798196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dedb37c6-8fad-4249-8c70-083fc8f5bbb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:54.798633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dedb37c6-8fad-4249-8c70-083fc8f5bbb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:54.799360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dedb37c6-8fad-4249-8c70-083fc8f5bbb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:54.799812 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dedb37c6-8fad-4249-8c70-083fc8f5bbb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:54.800260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dedb37c6-8fad-4249-8c70-083fc8f5bbb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:54.875645 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-dedb37c6-8fad-4249-8c70-083fc8f5bbb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:54.891840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dedb37c6-8fad-4249-8c70-083fc8f5bbb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.105666 Aug 07 01:29:54.892931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 857/1540] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:54 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:29:55.921174 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-45220961-d600-4bd3-ba13-5a086c234a9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:55.922289 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-45220961-d600-4bd3-ba13-5a086c234a9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:55.934742 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-45220961-d600-4bd3-ba13-5a086c234a9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:55.935727 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-45220961-d600-4bd3-ba13-5a086c234a9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:55.936458 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-45220961-d600-4bd3-ba13-5a086c234a9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:55.937738 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-45220961-d600-4bd3-ba13-5a086c234a9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:55.938550 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-45220961-d600-4bd3-ba13-5a086c234a9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:55.939304 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-45220961-d600-4bd3-ba13-5a086c234a9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:56.032142 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-45220961-d600-4bd3-ba13-5a086c234a9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:56.041984 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-45220961-d600-4bd3-ba13-5a086c234a9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.128397 Aug 07 01:29:56.042604 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 684/1541] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:55 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:57.066639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-984ed5b8-6618-4441-b44d-5c7ac5e770ea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:57.067544 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-984ed5b8-6618-4441-b44d-5c7ac5e770ea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:57.074902 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-984ed5b8-6618-4441-b44d-5c7ac5e770ea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:57.075448 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-984ed5b8-6618-4441-b44d-5c7ac5e770ea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:57.075929 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-984ed5b8-6618-4441-b44d-5c7ac5e770ea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:57.076742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-984ed5b8-6618-4441-b44d-5c7ac5e770ea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:57.077547 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-984ed5b8-6618-4441-b44d-5c7ac5e770ea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:57.078031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-984ed5b8-6618-4441-b44d-5c7ac5e770ea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:57.403819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-984ed5b8-6618-4441-b44d-5c7ac5e770ea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:57.413686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-984ed5b8-6618-4441-b44d-5c7ac5e770ea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.352153 Aug 07 01:29:57.414279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 858/1542] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:57 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:58.444031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f3acfd29-22a7-473a-851a-3d6d747e506e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:58.444965 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f3acfd29-22a7-473a-851a-3d6d747e506e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:58.457064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f3acfd29-22a7-473a-851a-3d6d747e506e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:58.457878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f3acfd29-22a7-473a-851a-3d6d747e506e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:58.458802 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f3acfd29-22a7-473a-851a-3d6d747e506e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:58.459962 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f3acfd29-22a7-473a-851a-3d6d747e506e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:58.460751 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f3acfd29-22a7-473a-851a-3d6d747e506e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:58.461486 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f3acfd29-22a7-473a-851a-3d6d747e506e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:58.555370 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f3acfd29-22a7-473a-851a-3d6d747e506e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:58.569627 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f3acfd29-22a7-473a-851a-3d6d747e506e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.129076 Aug 07 01:29:58.570859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 685/1543] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:58 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:29:59.596493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4e6592c1-0ab1-4faa-ad52-e6fdcb9e1c20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:29:59.597533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4e6592c1-0ab1-4faa-ad52-e6fdcb9e1c20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:29:59.609142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e6592c1-0ab1-4faa-ad52-e6fdcb9e1c20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:59.610102 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e6592c1-0ab1-4faa-ad52-e6fdcb9e1c20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:59.610778 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e6592c1-0ab1-4faa-ad52-e6fdcb9e1c20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:59.612101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e6592c1-0ab1-4faa-ad52-e6fdcb9e1c20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:29:59.612959 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e6592c1-0ab1-4faa-ad52-e6fdcb9e1c20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:29:59.613648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e6592c1-0ab1-4faa-ad52-e6fdcb9e1c20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:29:59.688350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4e6592c1-0ab1-4faa-ad52-e6fdcb9e1c20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:29:59.703347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4e6592c1-0ab1-4faa-ad52-e6fdcb9e1c20 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.113987 Aug 07 01:29:59.704418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 859/1544] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:29:59 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:00.738314 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d34ee3bc-fdae-4998-ac69-cd48595dcbf9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:00.739267 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d34ee3bc-fdae-4998-ac69-cd48595dcbf9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:00.746592 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d34ee3bc-fdae-4998-ac69-cd48595dcbf9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:00.747088 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d34ee3bc-fdae-4998-ac69-cd48595dcbf9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:00.747533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d34ee3bc-fdae-4998-ac69-cd48595dcbf9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:00.748295 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d34ee3bc-fdae-4998-ac69-cd48595dcbf9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:00.748758 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d34ee3bc-fdae-4998-ac69-cd48595dcbf9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:00.749190 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d34ee3bc-fdae-4998-ac69-cd48595dcbf9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:00.823378 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d34ee3bc-fdae-4998-ac69-cd48595dcbf9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:00.832379 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d34ee3bc-fdae-4998-ac69-cd48595dcbf9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098974 Aug 07 01:30:00.832976 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 686/1545] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:00 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:01.856957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e3fad160-aaf9-4181-96fd-104ad4858106 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:01.857656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e3fad160-aaf9-4181-96fd-104ad4858106 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:01.865150 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3fad160-aaf9-4181-96fd-104ad4858106 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:01.865612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3fad160-aaf9-4181-96fd-104ad4858106 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:01.866060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3fad160-aaf9-4181-96fd-104ad4858106 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:01.866801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3fad160-aaf9-4181-96fd-104ad4858106 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:01.867256 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3fad160-aaf9-4181-96fd-104ad4858106 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:01.867688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e3fad160-aaf9-4181-96fd-104ad4858106 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:01.940808 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e3fad160-aaf9-4181-96fd-104ad4858106 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:01.949909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e3fad160-aaf9-4181-96fd-104ad4858106 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097155 Aug 07 01:30:01.950525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 860/1546] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:01 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:02.973235 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6da2eb7f-495c-4850-8037-3ca55e64361f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:02.973843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6da2eb7f-495c-4850-8037-3ca55e64361f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:02.980470 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6da2eb7f-495c-4850-8037-3ca55e64361f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:02.980958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6da2eb7f-495c-4850-8037-3ca55e64361f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:02.981390 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6da2eb7f-495c-4850-8037-3ca55e64361f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:02.982097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6da2eb7f-495c-4850-8037-3ca55e64361f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:02.982547 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6da2eb7f-495c-4850-8037-3ca55e64361f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:02.982966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6da2eb7f-495c-4850-8037-3ca55e64361f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:03.048700 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6da2eb7f-495c-4850-8037-3ca55e64361f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:03.057424 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6da2eb7f-495c-4850-8037-3ca55e64361f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.088021 Aug 07 01:30:03.058025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 687/1547] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:02 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:04.085012 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6451deab-46c7-496d-ad3d-921d9c78b82b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:04.086025 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6451deab-46c7-496d-ad3d-921d9c78b82b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:04.093104 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6451deab-46c7-496d-ad3d-921d9c78b82b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:04.093602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6451deab-46c7-496d-ad3d-921d9c78b82b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:04.094035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6451deab-46c7-496d-ad3d-921d9c78b82b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:04.094762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6451deab-46c7-496d-ad3d-921d9c78b82b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:04.095219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6451deab-46c7-496d-ad3d-921d9c78b82b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:04.095640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6451deab-46c7-496d-ad3d-921d9c78b82b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:04.193064 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6451deab-46c7-496d-ad3d-921d9c78b82b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:04.206332 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6451deab-46c7-496d-ad3d-921d9c78b82b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.125745 Aug 07 01:30:04.206947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 861/1548] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:04 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:05.228595 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bae687d4-8b32-4118-9913-7d19ecdf2ddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:05.229444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bae687d4-8b32-4118-9913-7d19ecdf2ddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:05.236463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bae687d4-8b32-4118-9913-7d19ecdf2ddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:05.237015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bae687d4-8b32-4118-9913-7d19ecdf2ddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:05.237508 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bae687d4-8b32-4118-9913-7d19ecdf2ddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:05.238398 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bae687d4-8b32-4118-9913-7d19ecdf2ddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:05.239070 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bae687d4-8b32-4118-9913-7d19ecdf2ddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:05.239602 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bae687d4-8b32-4118-9913-7d19ecdf2ddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:05.402025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bae687d4-8b32-4118-9913-7d19ecdf2ddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:05.417213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bae687d4-8b32-4118-9913-7d19ecdf2ddb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.192221 Aug 07 01:30:05.418282 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 688/1549] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:05 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:06.451338 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-27c6c2a9-33d6-4635-a76f-e100cde57c5c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:06.451982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-27c6c2a9-33d6-4635-a76f-e100cde57c5c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:06.462385 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27c6c2a9-33d6-4635-a76f-e100cde57c5c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:06.463277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27c6c2a9-33d6-4635-a76f-e100cde57c5c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:06.463982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27c6c2a9-33d6-4635-a76f-e100cde57c5c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:06.464938 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27c6c2a9-33d6-4635-a76f-e100cde57c5c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:06.465456 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27c6c2a9-33d6-4635-a76f-e100cde57c5c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:06.465954 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27c6c2a9-33d6-4635-a76f-e100cde57c5c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:06.592151 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-27c6c2a9-33d6-4635-a76f-e100cde57c5c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:06.601059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-27c6c2a9-33d6-4635-a76f-e100cde57c5c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.153691 Aug 07 01:30:06.601664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 862/1550] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:06 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:07.633563 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-067bc59e-7fd6-4d39-b39a-502bc1798a4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:07.634391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-067bc59e-7fd6-4d39-b39a-502bc1798a4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:07.642096 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-067bc59e-7fd6-4d39-b39a-502bc1798a4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:07.642598 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-067bc59e-7fd6-4d39-b39a-502bc1798a4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:07.643064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-067bc59e-7fd6-4d39-b39a-502bc1798a4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:07.643783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-067bc59e-7fd6-4d39-b39a-502bc1798a4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:07.644279 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-067bc59e-7fd6-4d39-b39a-502bc1798a4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:07.644724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-067bc59e-7fd6-4d39-b39a-502bc1798a4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:07.714277 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-067bc59e-7fd6-4d39-b39a-502bc1798a4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:07.723140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-067bc59e-7fd6-4d39-b39a-502bc1798a4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.093813 Aug 07 01:30:07.723726 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 689/1551] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:07 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:08.018856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:30:08.019163 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:30:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:30:08.744511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-abcf0704-574f-4981-8fd7-d8fcb340771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:08.745132 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-abcf0704-574f-4981-8fd7-d8fcb340771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:08.752479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-abcf0704-574f-4981-8fd7-d8fcb340771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:08.752990 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-abcf0704-574f-4981-8fd7-d8fcb340771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:08.753439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-abcf0704-574f-4981-8fd7-d8fcb340771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:08.754178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-abcf0704-574f-4981-8fd7-d8fcb340771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:08.754641 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-abcf0704-574f-4981-8fd7-d8fcb340771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:08.755075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-abcf0704-574f-4981-8fd7-d8fcb340771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:08.821276 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-abcf0704-574f-4981-8fd7-d8fcb340771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:08.830052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-abcf0704-574f-4981-8fd7-d8fcb340771e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.089572 Aug 07 01:30:08.831169 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 863/1552] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:08 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:09.856911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-87aaef2a-5dcf-44e4-9dee-3d41aa16c518 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:09.858061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-87aaef2a-5dcf-44e4-9dee-3d41aa16c518 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:09.872018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-87aaef2a-5dcf-44e4-9dee-3d41aa16c518 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:09.872924 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-87aaef2a-5dcf-44e4-9dee-3d41aa16c518 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:09.873717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-87aaef2a-5dcf-44e4-9dee-3d41aa16c518 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:09.875042 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-87aaef2a-5dcf-44e4-9dee-3d41aa16c518 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:09.875867 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-87aaef2a-5dcf-44e4-9dee-3d41aa16c518 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:09.876676 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-87aaef2a-5dcf-44e4-9dee-3d41aa16c518 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:09.969632 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-87aaef2a-5dcf-44e4-9dee-3d41aa16c518 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:09.978940 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-87aaef2a-5dcf-44e4-9dee-3d41aa16c518 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.129770 Aug 07 01:30:09.979523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 690/1553] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:09 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:11.004145 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-207f3baa-99c5-413a-a749-f60f03c65c50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:11.005418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-207f3baa-99c5-413a-a749-f60f03c65c50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:11.017499 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-207f3baa-99c5-413a-a749-f60f03c65c50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:11.018413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-207f3baa-99c5-413a-a749-f60f03c65c50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:11.019247 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-207f3baa-99c5-413a-a749-f60f03c65c50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:11.020645 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-207f3baa-99c5-413a-a749-f60f03c65c50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:11.021503 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-207f3baa-99c5-413a-a749-f60f03c65c50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:11.022288 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-207f3baa-99c5-413a-a749-f60f03c65c50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:11.092652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-207f3baa-99c5-413a-a749-f60f03c65c50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:11.104313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-207f3baa-99c5-413a-a749-f60f03c65c50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.107430 Aug 07 01:30:11.105357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 864/1554] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:10 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:12.142053 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-962cb0e3-3232-4295-874e-cb265b05d7c3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:12.143973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-962cb0e3-3232-4295-874e-cb265b05d7c3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:12.153103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-962cb0e3-3232-4295-874e-cb265b05d7c3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:12.153991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-962cb0e3-3232-4295-874e-cb265b05d7c3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:12.154778 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-962cb0e3-3232-4295-874e-cb265b05d7c3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:12.156035 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-962cb0e3-3232-4295-874e-cb265b05d7c3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:12.156928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-962cb0e3-3232-4295-874e-cb265b05d7c3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:12.157697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-962cb0e3-3232-4295-874e-cb265b05d7c3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:12.250807 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-962cb0e3-3232-4295-874e-cb265b05d7c3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:12.265425 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-962cb0e3-3232-4295-874e-cb265b05d7c3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.128865 Aug 07 01:30:12.266376 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 691/1555] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:12 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:13.300936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0f9f2535-1547-4565-8eb3-c5f20f0f9a76 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:13.302028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0f9f2535-1547-4565-8eb3-c5f20f0f9a76 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:13.308772 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f9f2535-1547-4565-8eb3-c5f20f0f9a76 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:13.309329 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f9f2535-1547-4565-8eb3-c5f20f0f9a76 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:13.309880 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f9f2535-1547-4565-8eb3-c5f20f0f9a76 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:13.310757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f9f2535-1547-4565-8eb3-c5f20f0f9a76 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:13.311286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f9f2535-1547-4565-8eb3-c5f20f0f9a76 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:13.311760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f9f2535-1547-4565-8eb3-c5f20f0f9a76 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:13.394387 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0f9f2535-1547-4565-8eb3-c5f20f0f9a76 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:13.403601 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0f9f2535-1547-4565-8eb3-c5f20f0f9a76 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.108114 Aug 07 01:30:13.404191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 865/1556] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:13 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:14.427678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a2768e7c-ccb2-48e4-8bf7-e79955784170 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:14.428601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a2768e7c-ccb2-48e4-8bf7-e79955784170 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:14.435815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2768e7c-ccb2-48e4-8bf7-e79955784170 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:14.436352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2768e7c-ccb2-48e4-8bf7-e79955784170 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:14.436802 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2768e7c-ccb2-48e4-8bf7-e79955784170 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:14.437527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2768e7c-ccb2-48e4-8bf7-e79955784170 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:14.437992 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2768e7c-ccb2-48e4-8bf7-e79955784170 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:14.438426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2768e7c-ccb2-48e4-8bf7-e79955784170 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:14.509770 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a2768e7c-ccb2-48e4-8bf7-e79955784170 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:14.519533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a2768e7c-ccb2-48e4-8bf7-e79955784170 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097205 Aug 07 01:30:14.520144 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 692/1557] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:14 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:15.558768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7551d50c-afbe-46a9-aa4f-92f145bac0fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:15.559879 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7551d50c-afbe-46a9-aa4f-92f145bac0fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:15.570822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7551d50c-afbe-46a9-aa4f-92f145bac0fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:15.571645 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7551d50c-afbe-46a9-aa4f-92f145bac0fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:15.572425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7551d50c-afbe-46a9-aa4f-92f145bac0fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:15.573633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7551d50c-afbe-46a9-aa4f-92f145bac0fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:15.574395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7551d50c-afbe-46a9-aa4f-92f145bac0fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:15.575164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7551d50c-afbe-46a9-aa4f-92f145bac0fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:15.660730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7551d50c-afbe-46a9-aa4f-92f145bac0fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:15.669631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7551d50c-afbe-46a9-aa4f-92f145bac0fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.117836 Aug 07 01:30:15.670642 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 866/1558] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:15 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:16.693631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9a80e9f7-c073-4585-8970-cdf1ea07cd99 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:16.694414 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9a80e9f7-c073-4585-8970-cdf1ea07cd99 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:16.702033 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a80e9f7-c073-4585-8970-cdf1ea07cd99 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:16.702821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a80e9f7-c073-4585-8970-cdf1ea07cd99 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:16.703267 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a80e9f7-c073-4585-8970-cdf1ea07cd99 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:16.703996 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a80e9f7-c073-4585-8970-cdf1ea07cd99 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:16.704486 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a80e9f7-c073-4585-8970-cdf1ea07cd99 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:16.704918 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9a80e9f7-c073-4585-8970-cdf1ea07cd99 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:16.802360 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9a80e9f7-c073-4585-8970-cdf1ea07cd99 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:16.812053 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9a80e9f7-c073-4585-8970-cdf1ea07cd99 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.123538 Aug 07 01:30:16.812856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 693/1559] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:16 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:17.841081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-77ab28cd-9748-42c4-bc69-b106a2777c9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:17.842001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-77ab28cd-9748-42c4-bc69-b106a2777c9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:17.850408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77ab28cd-9748-42c4-bc69-b106a2777c9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:17.851048 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77ab28cd-9748-42c4-bc69-b106a2777c9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:17.851844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77ab28cd-9748-42c4-bc69-b106a2777c9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:17.852787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77ab28cd-9748-42c4-bc69-b106a2777c9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:17.853353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77ab28cd-9748-42c4-bc69-b106a2777c9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:17.853887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-77ab28cd-9748-42c4-bc69-b106a2777c9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:17.942846 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-77ab28cd-9748-42c4-bc69-b106a2777c9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:17.956062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-77ab28cd-9748-42c4-bc69-b106a2777c9b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.122254 Aug 07 01:30:17.957214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 867/1560] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:17 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:18.979440 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7aadc15c-1c26-4c70-9a86-992f1062fd5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:18.980579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7aadc15c-1c26-4c70-9a86-992f1062fd5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:18.988624 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7aadc15c-1c26-4c70-9a86-992f1062fd5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:18.989184 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7aadc15c-1c26-4c70-9a86-992f1062fd5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:18.989673 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7aadc15c-1c26-4c70-9a86-992f1062fd5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:18.991150 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7aadc15c-1c26-4c70-9a86-992f1062fd5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:18.991150 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7aadc15c-1c26-4c70-9a86-992f1062fd5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:18.991744 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7aadc15c-1c26-4c70-9a86-992f1062fd5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:19.077723 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7aadc15c-1c26-4c70-9a86-992f1062fd5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:19.093291 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7aadc15c-1c26-4c70-9a86-992f1062fd5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.118378 Aug 07 01:30:19.094354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 694/1561] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:18 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:20.132499 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-243e640f-80e0-4c64-8554-c9dccc4b8657 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:20.133241 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-243e640f-80e0-4c64-8554-c9dccc4b8657 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:20.136164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=28 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:30:20.146498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-243e640f-80e0-4c64-8554-c9dccc4b8657 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:20.147068 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-243e640f-80e0-4c64-8554-c9dccc4b8657 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:20.147513 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-243e640f-80e0-4c64-8554-c9dccc4b8657 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:20.148283 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-243e640f-80e0-4c64-8554-c9dccc4b8657 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:20.148747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-243e640f-80e0-4c64-8554-c9dccc4b8657 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:20.149377 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-243e640f-80e0-4c64-8554-c9dccc4b8657 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:20.152942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=84 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:30:20.233972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-243e640f-80e0-4c64-8554-c9dccc4b8657 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:20.244271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-243e640f-80e0-4c64-8554-c9dccc4b8657 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.116893 Aug 07 01:30:20.244884 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 868/1562] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:20 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:21.272534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-622c54b6-f612-4a49-9230-f0c7a31e7723 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:21.273651 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-622c54b6-f612-4a49-9230-f0c7a31e7723 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:21.276994 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=28 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:30:21.286409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-622c54b6-f612-4a49-9230-f0c7a31e7723 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:21.287108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-622c54b6-f612-4a49-9230-f0c7a31e7723 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:21.287648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-622c54b6-f612-4a49-9230-f0c7a31e7723 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:21.288586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-622c54b6-f612-4a49-9230-f0c7a31e7723 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:21.289317 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-622c54b6-f612-4a49-9230-f0c7a31e7723 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:21.289857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-622c54b6-f612-4a49-9230-f0c7a31e7723 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:21.297099 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=84 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:30:21.371673 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-622c54b6-f612-4a49-9230-f0c7a31e7723 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:21.381198 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-622c54b6-f612-4a49-9230-f0c7a31e7723 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.117034 Aug 07 01:30:21.381796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 695/1563] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:21 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:22.407309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0e5e88f0-bbbf-4c63-a785-cc8311afbe92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:22.408944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0e5e88f0-bbbf-4c63-a785-cc8311afbe92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:22.421324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e5e88f0-bbbf-4c63-a785-cc8311afbe92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:22.422161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e5e88f0-bbbf-4c63-a785-cc8311afbe92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:22.422931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e5e88f0-bbbf-4c63-a785-cc8311afbe92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:22.424717 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e5e88f0-bbbf-4c63-a785-cc8311afbe92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:22.425539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e5e88f0-bbbf-4c63-a785-cc8311afbe92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:22.426294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0e5e88f0-bbbf-4c63-a785-cc8311afbe92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:22.535646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0e5e88f0-bbbf-4c63-a785-cc8311afbe92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:22.547682 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0e5e88f0-bbbf-4c63-a785-cc8311afbe92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.147260 Aug 07 01:30:22.549026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 869/1564] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:22 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:23.584473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3172d8d6-bb35-4fea-bf9d-0e87fbcd591d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:23.585337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3172d8d6-bb35-4fea-bf9d-0e87fbcd591d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:23.593119 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3172d8d6-bb35-4fea-bf9d-0e87fbcd591d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:23.593682 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3172d8d6-bb35-4fea-bf9d-0e87fbcd591d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:23.594117 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3172d8d6-bb35-4fea-bf9d-0e87fbcd591d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:23.594889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3172d8d6-bb35-4fea-bf9d-0e87fbcd591d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:23.595352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3172d8d6-bb35-4fea-bf9d-0e87fbcd591d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:23.595774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3172d8d6-bb35-4fea-bf9d-0e87fbcd591d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:23.679127 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3172d8d6-bb35-4fea-bf9d-0e87fbcd591d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:23.692023 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3172d8d6-bb35-4fea-bf9d-0e87fbcd591d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.112460 Aug 07 01:30:23.692991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 696/1565] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:23 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:24.717166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-569fe5ca-2f19-4e38-b5c3-57ae255ee702 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:24.717758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-569fe5ca-2f19-4e38-b5c3-57ae255ee702 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:24.726491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-569fe5ca-2f19-4e38-b5c3-57ae255ee702 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:24.726994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-569fe5ca-2f19-4e38-b5c3-57ae255ee702 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:24.727530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-569fe5ca-2f19-4e38-b5c3-57ae255ee702 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:24.728459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-569fe5ca-2f19-4e38-b5c3-57ae255ee702 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:24.728925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-569fe5ca-2f19-4e38-b5c3-57ae255ee702 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:24.729365 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-569fe5ca-2f19-4e38-b5c3-57ae255ee702 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:24.803891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-569fe5ca-2f19-4e38-b5c3-57ae255ee702 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:24.812724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-569fe5ca-2f19-4e38-b5c3-57ae255ee702 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.099547 Aug 07 01:30:24.813506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 870/1566] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:24 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:25.839398 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-48efd0de-fb93-4213-b70c-087905c60c77 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:25.840540 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-48efd0de-fb93-4213-b70c-087905c60c77 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:25.852739 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-48efd0de-fb93-4213-b70c-087905c60c77 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:25.853632 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-48efd0de-fb93-4213-b70c-087905c60c77 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:25.854420 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-48efd0de-fb93-4213-b70c-087905c60c77 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:25.855711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-48efd0de-fb93-4213-b70c-087905c60c77 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:25.856589 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-48efd0de-fb93-4213-b70c-087905c60c77 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:25.857371 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-48efd0de-fb93-4213-b70c-087905c60c77 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:25.937591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-48efd0de-fb93-4213-b70c-087905c60c77 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:25.947017 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-48efd0de-fb93-4213-b70c-087905c60c77 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.114824 Aug 07 01:30:25.947622 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 697/1567] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:25 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:26.972043 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e9d04c43-a812-4bd1-afbe-1b929494281e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:26.972720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e9d04c43-a812-4bd1-afbe-1b929494281e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:26.979744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9d04c43-a812-4bd1-afbe-1b929494281e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:26.980298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9d04c43-a812-4bd1-afbe-1b929494281e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:26.980728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9d04c43-a812-4bd1-afbe-1b929494281e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:26.981458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9d04c43-a812-4bd1-afbe-1b929494281e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:26.981930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9d04c43-a812-4bd1-afbe-1b929494281e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:26.982448 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9d04c43-a812-4bd1-afbe-1b929494281e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:27.055475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e9d04c43-a812-4bd1-afbe-1b929494281e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:27.071175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e9d04c43-a812-4bd1-afbe-1b929494281e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.103013 Aug 07 01:30:27.072461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 871/1568] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:26 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:28.099848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1011c701-d998-4881-83d0-e6956bd84bfd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:28.100997 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1011c701-d998-4881-83d0-e6956bd84bfd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:28.113212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1011c701-d998-4881-83d0-e6956bd84bfd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:28.114071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1011c701-d998-4881-83d0-e6956bd84bfd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:28.114858 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1011c701-d998-4881-83d0-e6956bd84bfd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:28.116124 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1011c701-d998-4881-83d0-e6956bd84bfd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:28.116982 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1011c701-d998-4881-83d0-e6956bd84bfd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:28.117752 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1011c701-d998-4881-83d0-e6956bd84bfd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:28.217343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1011c701-d998-4881-83d0-e6956bd84bfd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:28.232788 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1011c701-d998-4881-83d0-e6956bd84bfd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.139814 Aug 07 01:30:28.233860 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 698/1569] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:28 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:28.423695 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:30:28.424033 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:30:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:30:29.275711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-910f321e-ee33-48cf-89cf-5bd73527fc2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:29.276907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-910f321e-ee33-48cf-89cf-5bd73527fc2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:29.290839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-910f321e-ee33-48cf-89cf-5bd73527fc2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:29.291787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-910f321e-ee33-48cf-89cf-5bd73527fc2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:29.292624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-910f321e-ee33-48cf-89cf-5bd73527fc2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:29.293928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-910f321e-ee33-48cf-89cf-5bd73527fc2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:29.294732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-910f321e-ee33-48cf-89cf-5bd73527fc2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:29.295486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-910f321e-ee33-48cf-89cf-5bd73527fc2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:29.404095 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-910f321e-ee33-48cf-89cf-5bd73527fc2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:29.413731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-910f321e-ee33-48cf-89cf-5bd73527fc2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.146622 Aug 07 01:30:29.414760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 872/1570] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:29 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:30.450515 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-672e3a8d-de0a-4115-a4e3-086f9d045e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:30.451191 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-672e3a8d-de0a-4115-a4e3-086f9d045e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:30.459025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-672e3a8d-de0a-4115-a4e3-086f9d045e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:30.459518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-672e3a8d-de0a-4115-a4e3-086f9d045e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:30.459968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-672e3a8d-de0a-4115-a4e3-086f9d045e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:30.460732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-672e3a8d-de0a-4115-a4e3-086f9d045e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:30.461203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-672e3a8d-de0a-4115-a4e3-086f9d045e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:30.461639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-672e3a8d-de0a-4115-a4e3-086f9d045e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:30.550792 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-672e3a8d-de0a-4115-a4e3-086f9d045e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:30.559983 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-672e3a8d-de0a-4115-a4e3-086f9d045e8b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.113298 Aug 07 01:30:30.560616 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 699/1571] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:30 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:31.587781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-416a8df5-13d7-4b6c-aa1f-61a5e63a80a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:31.588497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-416a8df5-13d7-4b6c-aa1f-61a5e63a80a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:31.595907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-416a8df5-13d7-4b6c-aa1f-61a5e63a80a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:31.596612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-416a8df5-13d7-4b6c-aa1f-61a5e63a80a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:31.597174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-416a8df5-13d7-4b6c-aa1f-61a5e63a80a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:31.598029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-416a8df5-13d7-4b6c-aa1f-61a5e63a80a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:31.598601 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-416a8df5-13d7-4b6c-aa1f-61a5e63a80a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:31.599147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-416a8df5-13d7-4b6c-aa1f-61a5e63a80a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:31.669932 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-416a8df5-13d7-4b6c-aa1f-61a5e63a80a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:31.678917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-416a8df5-13d7-4b6c-aa1f-61a5e63a80a3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.095563 Aug 07 01:30:31.679511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 873/1572] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:31 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:32.702779 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-22c23989-14b4-4634-9be2-e4dc4662e869 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:32.703387 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-22c23989-14b4-4634-9be2-e4dc4662e869 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:32.711634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-22c23989-14b4-4634-9be2-e4dc4662e869 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:32.712126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-22c23989-14b4-4634-9be2-e4dc4662e869 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:32.712625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-22c23989-14b4-4634-9be2-e4dc4662e869 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:32.713370 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-22c23989-14b4-4634-9be2-e4dc4662e869 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:32.713831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-22c23989-14b4-4634-9be2-e4dc4662e869 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:32.714271 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-22c23989-14b4-4634-9be2-e4dc4662e869 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:32.806706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-22c23989-14b4-4634-9be2-e4dc4662e869 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:32.816302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-22c23989-14b4-4634-9be2-e4dc4662e869 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.118350 Aug 07 01:30:32.816909 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 700/1573] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:32 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:33.840572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ad2b0af5-dc87-4539-bb41-d7ca15c1ea5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:33.841295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ad2b0af5-dc87-4539-bb41-d7ca15c1ea5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:33.849022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ad2b0af5-dc87-4539-bb41-d7ca15c1ea5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:33.849724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ad2b0af5-dc87-4539-bb41-d7ca15c1ea5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:33.850222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ad2b0af5-dc87-4539-bb41-d7ca15c1ea5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:33.851099 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ad2b0af5-dc87-4539-bb41-d7ca15c1ea5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:33.851690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ad2b0af5-dc87-4539-bb41-d7ca15c1ea5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:33.852177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ad2b0af5-dc87-4539-bb41-d7ca15c1ea5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:33.933261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ad2b0af5-dc87-4539-bb41-d7ca15c1ea5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:33.951069 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ad2b0af5-dc87-4539-bb41-d7ca15c1ea5f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.114596 Aug 07 01:30:33.952281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 874/1574] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:33 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:34.995754 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2d160a55-52bd-4fab-9986-41f51fe15961 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:34.999186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2d160a55-52bd-4fab-9986-41f51fe15961 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:35.005905 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2d160a55-52bd-4fab-9986-41f51fe15961 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:35.006556 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2d160a55-52bd-4fab-9986-41f51fe15961 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:35.007097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2d160a55-52bd-4fab-9986-41f51fe15961 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:35.008048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2d160a55-52bd-4fab-9986-41f51fe15961 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:35.008651 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2d160a55-52bd-4fab-9986-41f51fe15961 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:35.009179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2d160a55-52bd-4fab-9986-41f51fe15961 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:35.086979 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2d160a55-52bd-4fab-9986-41f51fe15961 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:35.097073 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2d160a55-52bd-4fab-9986-41f51fe15961 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.106262 Aug 07 01:30:35.097677 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 701/1575] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:34 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:36.132811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-68024c6c-fad8-436a-b22e-7b5c8a3e75a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:36.133668 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-68024c6c-fad8-436a-b22e-7b5c8a3e75a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:36.141748 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68024c6c-fad8-436a-b22e-7b5c8a3e75a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:36.143006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68024c6c-fad8-436a-b22e-7b5c8a3e75a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:36.143408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68024c6c-fad8-436a-b22e-7b5c8a3e75a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:36.144217 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68024c6c-fad8-436a-b22e-7b5c8a3e75a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:36.144845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68024c6c-fad8-436a-b22e-7b5c8a3e75a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:36.145331 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68024c6c-fad8-436a-b22e-7b5c8a3e75a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:36.218106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-68024c6c-fad8-436a-b22e-7b5c8a3e75a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:36.227940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-68024c6c-fad8-436a-b22e-7b5c8a3e75a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.099303 Aug 07 01:30:36.228569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 875/1576] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:36 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:37.253363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-96c6915b-bbf6-4cf4-871b-c32256050195 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:37.254325 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-96c6915b-bbf6-4cf4-871b-c32256050195 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:37.263090 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-96c6915b-bbf6-4cf4-871b-c32256050195 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:37.263768 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-96c6915b-bbf6-4cf4-871b-c32256050195 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:37.264218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-96c6915b-bbf6-4cf4-871b-c32256050195 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:37.265082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-96c6915b-bbf6-4cf4-871b-c32256050195 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:37.265716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-96c6915b-bbf6-4cf4-871b-c32256050195 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:37.266399 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-96c6915b-bbf6-4cf4-871b-c32256050195 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:37.361572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-96c6915b-bbf6-4cf4-871b-c32256050195 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:37.372930 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-96c6915b-bbf6-4cf4-871b-c32256050195 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.125994 Aug 07 01:30:37.373769 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 702/1577] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:37 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:38.398499 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b02aac41-d73b-4636-ae72-983954155425 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:38.399194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b02aac41-d73b-4636-ae72-983954155425 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:38.410787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b02aac41-d73b-4636-ae72-983954155425 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:38.411669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b02aac41-d73b-4636-ae72-983954155425 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:38.412481 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b02aac41-d73b-4636-ae72-983954155425 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:38.413796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b02aac41-d73b-4636-ae72-983954155425 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:38.414627 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b02aac41-d73b-4636-ae72-983954155425 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:38.414627 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b02aac41-d73b-4636-ae72-983954155425 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:38.527167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b02aac41-d73b-4636-ae72-983954155425 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:38.537548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b02aac41-d73b-4636-ae72-983954155425 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.145920 Aug 07 01:30:38.538375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 876/1578] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:38 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:39.567715 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-43d4eb95-f064-40a6-bf1b-8f5e3ddf2eda tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:39.568883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-43d4eb95-f064-40a6-bf1b-8f5e3ddf2eda tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:39.579106 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-43d4eb95-f064-40a6-bf1b-8f5e3ddf2eda tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:39.580416 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-43d4eb95-f064-40a6-bf1b-8f5e3ddf2eda tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:39.581017 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-43d4eb95-f064-40a6-bf1b-8f5e3ddf2eda tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:39.582228 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-43d4eb95-f064-40a6-bf1b-8f5e3ddf2eda tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:39.582842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-43d4eb95-f064-40a6-bf1b-8f5e3ddf2eda tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:39.583434 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-43d4eb95-f064-40a6-bf1b-8f5e3ddf2eda tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:39.694189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-43d4eb95-f064-40a6-bf1b-8f5e3ddf2eda tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:39.707427 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-43d4eb95-f064-40a6-bf1b-8f5e3ddf2eda tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.147238 Aug 07 01:30:39.708950 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 703/1579] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:39 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:40.734077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-71f70734-37ec-4373-9f07-3af059988fec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:40.734901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-71f70734-37ec-4373-9f07-3af059988fec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:40.743464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71f70734-37ec-4373-9f07-3af059988fec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:40.744351 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71f70734-37ec-4373-9f07-3af059988fec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:40.744821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71f70734-37ec-4373-9f07-3af059988fec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:40.745593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71f70734-37ec-4373-9f07-3af059988fec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:40.746363 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71f70734-37ec-4373-9f07-3af059988fec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:40.746931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71f70734-37ec-4373-9f07-3af059988fec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:40.852478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-71f70734-37ec-4373-9f07-3af059988fec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:40.869609 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-71f70734-37ec-4373-9f07-3af059988fec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.140805 Aug 07 01:30:40.870247 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 877/1580] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:40 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:41.910516 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e426c593-8711-4d82-9dbb-48c03872e943 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:41.911706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e426c593-8711-4d82-9dbb-48c03872e943 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:41.924123 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e426c593-8711-4d82-9dbb-48c03872e943 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:41.925043 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e426c593-8711-4d82-9dbb-48c03872e943 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:41.925859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e426c593-8711-4d82-9dbb-48c03872e943 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:41.927185 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e426c593-8711-4d82-9dbb-48c03872e943 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:41.928036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e426c593-8711-4d82-9dbb-48c03872e943 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:41.928844 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e426c593-8711-4d82-9dbb-48c03872e943 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:42.047212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e426c593-8711-4d82-9dbb-48c03872e943 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:42.060776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e426c593-8711-4d82-9dbb-48c03872e943 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.165760 Aug 07 01:30:42.061677 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 704/1581] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:41 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:43.093760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1416fe66-0536-4e31-906d-05fcc9ca72b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:43.094883 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1416fe66-0536-4e31-906d-05fcc9ca72b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:43.108479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1416fe66-0536-4e31-906d-05fcc9ca72b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:43.109363 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1416fe66-0536-4e31-906d-05fcc9ca72b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:43.110124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1416fe66-0536-4e31-906d-05fcc9ca72b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:43.112056 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1416fe66-0536-4e31-906d-05fcc9ca72b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:43.112943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1416fe66-0536-4e31-906d-05fcc9ca72b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:43.113710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1416fe66-0536-4e31-906d-05fcc9ca72b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:43.203810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1416fe66-0536-4e31-906d-05fcc9ca72b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:43.224341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1416fe66-0536-4e31-906d-05fcc9ca72b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.137460 Aug 07 01:30:43.225991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 878/1582] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:43 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:44.271083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-97be388a-a62e-43f3-a74f-9549d4fad9d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:44.279092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-97be388a-a62e-43f3-a74f-9549d4fad9d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:44.290256 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-97be388a-a62e-43f3-a74f-9549d4fad9d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:44.290942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-97be388a-a62e-43f3-a74f-9549d4fad9d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:44.291334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-97be388a-a62e-43f3-a74f-9549d4fad9d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:44.292931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-97be388a-a62e-43f3-a74f-9549d4fad9d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:44.292931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-97be388a-a62e-43f3-a74f-9549d4fad9d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:44.293278 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-97be388a-a62e-43f3-a74f-9549d4fad9d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:44.386612 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-97be388a-a62e-43f3-a74f-9549d4fad9d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:44.399794 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-97be388a-a62e-43f3-a74f-9549d4fad9d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.141850 Aug 07 01:30:44.400967 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 705/1583] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:44 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:45.424509 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cd1986bf-04e1-48d5-bcc6-714481f3de88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:45.425402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cd1986bf-04e1-48d5-bcc6-714481f3de88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:45.436198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd1986bf-04e1-48d5-bcc6-714481f3de88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:45.437049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd1986bf-04e1-48d5-bcc6-714481f3de88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:45.437759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd1986bf-04e1-48d5-bcc6-714481f3de88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:45.439508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd1986bf-04e1-48d5-bcc6-714481f3de88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:45.440267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd1986bf-04e1-48d5-bcc6-714481f3de88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:45.441105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd1986bf-04e1-48d5-bcc6-714481f3de88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:45.561309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-cd1986bf-04e1-48d5-bcc6-714481f3de88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:45.573884 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cd1986bf-04e1-48d5-bcc6-714481f3de88 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.154489 Aug 07 01:30:45.574809 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 879/1584] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:45 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:46.606104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-acffa4f5-24a3-4cb7-8567-dff309a19564 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:46.606963 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-acffa4f5-24a3-4cb7-8567-dff309a19564 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:46.618656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acffa4f5-24a3-4cb7-8567-dff309a19564 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:46.619599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acffa4f5-24a3-4cb7-8567-dff309a19564 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:46.620082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acffa4f5-24a3-4cb7-8567-dff309a19564 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:46.621313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acffa4f5-24a3-4cb7-8567-dff309a19564 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:46.622033 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acffa4f5-24a3-4cb7-8567-dff309a19564 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:46.622746 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-acffa4f5-24a3-4cb7-8567-dff309a19564 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:46.719605 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-acffa4f5-24a3-4cb7-8567-dff309a19564 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:46.730455 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-acffa4f5-24a3-4cb7-8567-dff309a19564 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.128374 Aug 07 01:30:46.731409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 706/1585] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:46 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:47.760102 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8db06b43-1938-4b7c-9fd4-960f5c1f371c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:47.761005 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8db06b43-1938-4b7c-9fd4-960f5c1f371c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:47.773366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8db06b43-1938-4b7c-9fd4-960f5c1f371c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:47.774245 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8db06b43-1938-4b7c-9fd4-960f5c1f371c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:47.774759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8db06b43-1938-4b7c-9fd4-960f5c1f371c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:47.775973 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8db06b43-1938-4b7c-9fd4-960f5c1f371c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:47.776704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8db06b43-1938-4b7c-9fd4-960f5c1f371c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:47.777529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8db06b43-1938-4b7c-9fd4-960f5c1f371c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:47.870031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8db06b43-1938-4b7c-9fd4-960f5c1f371c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:47.880003 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8db06b43-1938-4b7c-9fd4-960f5c1f371c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.127090 Aug 07 01:30:47.880433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 880/1586] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:47 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:48.916064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5acca3c6-6121-4969-84fc-f370558a8c3e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:48.917262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5acca3c6-6121-4969-84fc-f370558a8c3e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:48.930296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5acca3c6-6121-4969-84fc-f370558a8c3e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:48.931230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5acca3c6-6121-4969-84fc-f370558a8c3e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:48.932301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5acca3c6-6121-4969-84fc-f370558a8c3e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:48.933634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5acca3c6-6121-4969-84fc-f370558a8c3e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:48.934472 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5acca3c6-6121-4969-84fc-f370558a8c3e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:48.935270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5acca3c6-6121-4969-84fc-f370558a8c3e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:49.030318 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5acca3c6-6121-4969-84fc-f370558a8c3e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:49.041464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5acca3c6-6121-4969-84fc-f370558a8c3e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.132522 Aug 07 01:30:49.042256 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 707/1587] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:48 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:30:50.075843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-73eab2d1-2098-4e10-9101-52c17a6b786c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:50.076555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-73eab2d1-2098-4e10-9101-52c17a6b786c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:50.084213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73eab2d1-2098-4e10-9101-52c17a6b786c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:50.084744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73eab2d1-2098-4e10-9101-52c17a6b786c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:50.085191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73eab2d1-2098-4e10-9101-52c17a6b786c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:50.085937 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73eab2d1-2098-4e10-9101-52c17a6b786c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:50.086397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73eab2d1-2098-4e10-9101-52c17a6b786c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:50.086839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73eab2d1-2098-4e10-9101-52c17a6b786c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:50.175236 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-73eab2d1-2098-4e10-9101-52c17a6b786c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:50.178266 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=82 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:30:50.191004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-73eab2d1-2098-4e10-9101-52c17a6b786c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.117924 Aug 07 01:30:50.192490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 881/1588] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:50 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:51.226228 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-02df136d-83b0-4855-a031-65a5e0a66754 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:51.227023 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-02df136d-83b0-4855-a031-65a5e0a66754 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:51.233351 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02df136d-83b0-4855-a031-65a5e0a66754 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:51.233842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02df136d-83b0-4855-a031-65a5e0a66754 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:51.234274 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02df136d-83b0-4855-a031-65a5e0a66754 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:51.234996 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02df136d-83b0-4855-a031-65a5e0a66754 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:51.235454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02df136d-83b0-4855-a031-65a5e0a66754 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:51.236540 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-02df136d-83b0-4855-a031-65a5e0a66754 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:51.303860 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-02df136d-83b0-4855-a031-65a5e0a66754 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:51.313815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-02df136d-83b0-4855-a031-65a5e0a66754 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.091341 Aug 07 01:30:51.314770 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 708/1589] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:51 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:52.335307 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0a38961c-e825-4be8-9280-47edf0139097 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:52.336015 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0a38961c-e825-4be8-9280-47edf0139097 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:52.338315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=28 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:30:52.343580 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0a38961c-e825-4be8-9280-47edf0139097 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:52.344061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0a38961c-e825-4be8-9280-47edf0139097 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:52.344530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0a38961c-e825-4be8-9280-47edf0139097 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:52.345338 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0a38961c-e825-4be8-9280-47edf0139097 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:52.345793 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0a38961c-e825-4be8-9280-47edf0139097 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:52.346211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0a38961c-e825-4be8-9280-47edf0139097 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:52.423300 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0a38961c-e825-4be8-9280-47edf0139097 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:52.432755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0a38961c-e825-4be8-9280-47edf0139097 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.101371 Aug 07 01:30:52.433352 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 882/1590] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:52 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:53.455945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ce5cd12f-eb55-4c9a-ab3d-7952b63f0d5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:53.456616 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ce5cd12f-eb55-4c9a-ab3d-7952b63f0d5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:53.459101 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=28 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:30:53.464746 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce5cd12f-eb55-4c9a-ab3d-7952b63f0d5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:53.465237 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce5cd12f-eb55-4c9a-ab3d-7952b63f0d5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:53.465668 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce5cd12f-eb55-4c9a-ab3d-7952b63f0d5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:53.466377 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce5cd12f-eb55-4c9a-ab3d-7952b63f0d5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:53.466831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce5cd12f-eb55-4c9a-ab3d-7952b63f0d5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:53.467252 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ce5cd12f-eb55-4c9a-ab3d-7952b63f0d5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:53.470601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=84 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:30:53.551625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ce5cd12f-eb55-4c9a-ab3d-7952b63f0d5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:53.560809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ce5cd12f-eb55-4c9a-ab3d-7952b63f0d5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.110050 Aug 07 01:30:53.561526 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 709/1591] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:53 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:54.598721 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-735cf46a-901c-47c3-bfa9-e847f8fc1155 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:54.599351 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-735cf46a-901c-47c3-bfa9-e847f8fc1155 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:54.607046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735cf46a-901c-47c3-bfa9-e847f8fc1155 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:54.607534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735cf46a-901c-47c3-bfa9-e847f8fc1155 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:54.607972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735cf46a-901c-47c3-bfa9-e847f8fc1155 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:54.608737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735cf46a-901c-47c3-bfa9-e847f8fc1155 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:54.609184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735cf46a-901c-47c3-bfa9-e847f8fc1155 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:54.609609 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735cf46a-901c-47c3-bfa9-e847f8fc1155 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:54.681991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-735cf46a-901c-47c3-bfa9-e847f8fc1155 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:54.692625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-735cf46a-901c-47c3-bfa9-e847f8fc1155 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098105 Aug 07 01:30:54.693219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 883/1592] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:54 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:55.729708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2a60277d-ba78-4b91-b53c-5bb48401645d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:55.730307 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2a60277d-ba78-4b91-b53c-5bb48401645d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:55.737915 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a60277d-ba78-4b91-b53c-5bb48401645d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:55.738402 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a60277d-ba78-4b91-b53c-5bb48401645d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:55.739048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a60277d-ba78-4b91-b53c-5bb48401645d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:55.739976 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a60277d-ba78-4b91-b53c-5bb48401645d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:55.740490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a60277d-ba78-4b91-b53c-5bb48401645d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:55.740917 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a60277d-ba78-4b91-b53c-5bb48401645d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:55.815392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2a60277d-ba78-4b91-b53c-5bb48401645d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:55.824436 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2a60277d-ba78-4b91-b53c-5bb48401645d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098803 Aug 07 01:30:55.825024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 710/1593] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:55 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:56.846106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-76376e93-bbd3-4207-8c58-444e6246c096 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:56.846907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-76376e93-bbd3-4207-8c58-444e6246c096 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:56.854336 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76376e93-bbd3-4207-8c58-444e6246c096 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:56.854863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76376e93-bbd3-4207-8c58-444e6246c096 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:56.855317 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76376e93-bbd3-4207-8c58-444e6246c096 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:56.856054 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76376e93-bbd3-4207-8c58-444e6246c096 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:56.856542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76376e93-bbd3-4207-8c58-444e6246c096 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:56.857015 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76376e93-bbd3-4207-8c58-444e6246c096 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:56.939787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-76376e93-bbd3-4207-8c58-444e6246c096 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:56.948890 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-76376e93-bbd3-4207-8c58-444e6246c096 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.106985 Aug 07 01:30:56.949550 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 884/1594] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:56 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:57.971856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-01b0d838-b3f0-4a92-8f0f-33a94cf56fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:57.972614 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-01b0d838-b3f0-4a92-8f0f-33a94cf56fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:57.979541 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-01b0d838-b3f0-4a92-8f0f-33a94cf56fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:57.980027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-01b0d838-b3f0-4a92-8f0f-33a94cf56fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:57.980625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-01b0d838-b3f0-4a92-8f0f-33a94cf56fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:57.981351 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-01b0d838-b3f0-4a92-8f0f-33a94cf56fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:57.981686 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-01b0d838-b3f0-4a92-8f0f-33a94cf56fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:57.982112 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-01b0d838-b3f0-4a92-8f0f-33a94cf56fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:58.056894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-01b0d838-b3f0-4a92-8f0f-33a94cf56fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:58.066961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-01b0d838-b3f0-4a92-8f0f-33a94cf56fac tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.099858 Aug 07 01:30:58.068024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 711/1595] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:57 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:30:59.103019 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-881142c4-40e0-448b-a61f-944a93d0166c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:30:59.103769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-881142c4-40e0-448b-a61f-944a93d0166c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:30:59.112387 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-881142c4-40e0-448b-a61f-944a93d0166c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:59.113126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-881142c4-40e0-448b-a61f-944a93d0166c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:59.113713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-881142c4-40e0-448b-a61f-944a93d0166c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:59.114619 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-881142c4-40e0-448b-a61f-944a93d0166c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:30:59.115262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-881142c4-40e0-448b-a61f-944a93d0166c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:30:59.115799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-881142c4-40e0-448b-a61f-944a93d0166c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:30:59.191071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-881142c4-40e0-448b-a61f-944a93d0166c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:30:59.202015 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-881142c4-40e0-448b-a61f-944a93d0166c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.103102 Aug 07 01:30:59.202621 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 885/1596] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:30:59 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:00.223556 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d8b62a9d-704c-4784-9170-f3b6ef72a99f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:00.224472 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d8b62a9d-704c-4784-9170-f3b6ef72a99f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:00.231819 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8b62a9d-704c-4784-9170-f3b6ef72a99f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:00.232342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8b62a9d-704c-4784-9170-f3b6ef72a99f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:00.232785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8b62a9d-704c-4784-9170-f3b6ef72a99f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:00.233506 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8b62a9d-704c-4784-9170-f3b6ef72a99f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:00.233959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8b62a9d-704c-4784-9170-f3b6ef72a99f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:00.234384 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8b62a9d-704c-4784-9170-f3b6ef72a99f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:00.306940 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d8b62a9d-704c-4784-9170-f3b6ef72a99f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:00.317496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d8b62a9d-704c-4784-9170-f3b6ef72a99f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098030 Aug 07 01:31:00.318076 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 712/1597] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:00 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:01.351011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4f6566f0-a001-44b6-8c5e-0a6a1732639a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:01.351823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4f6566f0-a001-44b6-8c5e-0a6a1732639a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:01.358587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4f6566f0-a001-44b6-8c5e-0a6a1732639a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:01.359073 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4f6566f0-a001-44b6-8c5e-0a6a1732639a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:01.359514 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4f6566f0-a001-44b6-8c5e-0a6a1732639a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:01.360291 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4f6566f0-a001-44b6-8c5e-0a6a1732639a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:01.360749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4f6566f0-a001-44b6-8c5e-0a6a1732639a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:01.361175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4f6566f0-a001-44b6-8c5e-0a6a1732639a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:01.435149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4f6566f0-a001-44b6-8c5e-0a6a1732639a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:01.444039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4f6566f0-a001-44b6-8c5e-0a6a1732639a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097506 Aug 07 01:31:01.444674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 886/1598] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:01 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:02.467159 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1084fda7-8280-4a84-94de-ca97e330b450 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:02.467806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1084fda7-8280-4a84-94de-ca97e330b450 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:02.476850 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1084fda7-8280-4a84-94de-ca97e330b450 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:02.477355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1084fda7-8280-4a84-94de-ca97e330b450 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:02.477817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1084fda7-8280-4a84-94de-ca97e330b450 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:02.478555 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1084fda7-8280-4a84-94de-ca97e330b450 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:02.479015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1084fda7-8280-4a84-94de-ca97e330b450 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:02.479448 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1084fda7-8280-4a84-94de-ca97e330b450 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:02.547624 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1084fda7-8280-4a84-94de-ca97e330b450 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:02.558074 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1084fda7-8280-4a84-94de-ca97e330b450 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.095201 Aug 07 01:31:02.559074 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 713/1599] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:02 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:03.583768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c122a511-38fd-449a-8f14-0842a4f5e2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:03.584757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c122a511-38fd-449a-8f14-0842a4f5e2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:03.591534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c122a511-38fd-449a-8f14-0842a4f5e2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:03.592065 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c122a511-38fd-449a-8f14-0842a4f5e2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:03.592594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c122a511-38fd-449a-8f14-0842a4f5e2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:03.593426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c122a511-38fd-449a-8f14-0842a4f5e2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:03.593923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c122a511-38fd-449a-8f14-0842a4f5e2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:03.594577 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c122a511-38fd-449a-8f14-0842a4f5e2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:03.663770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c122a511-38fd-449a-8f14-0842a4f5e2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:03.672733 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c122a511-38fd-449a-8f14-0842a4f5e2e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.095934 Aug 07 01:31:03.673910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 887/1600] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:03 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:04.695477 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7e532606-2a1e-442f-9886-c6d11c0ad955 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:04.696077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7e532606-2a1e-442f-9886-c6d11c0ad955 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:04.704206 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7e532606-2a1e-442f-9886-c6d11c0ad955 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:04.704955 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7e532606-2a1e-442f-9886-c6d11c0ad955 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:04.705505 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7e532606-2a1e-442f-9886-c6d11c0ad955 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:04.706461 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7e532606-2a1e-442f-9886-c6d11c0ad955 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:04.707238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7e532606-2a1e-442f-9886-c6d11c0ad955 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:04.707831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7e532606-2a1e-442f-9886-c6d11c0ad955 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:04.780400 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7e532606-2a1e-442f-9886-c6d11c0ad955 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:04.789700 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7e532606-2a1e-442f-9886-c6d11c0ad955 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098094 Aug 07 01:31:04.790177 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 714/1601] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:04 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:05.812022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-94326ead-b2df-4e49-b1d7-357cc1d6d42f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:05.812755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-94326ead-b2df-4e49-b1d7-357cc1d6d42f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:05.820526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94326ead-b2df-4e49-b1d7-357cc1d6d42f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:05.821020 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94326ead-b2df-4e49-b1d7-357cc1d6d42f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:05.821492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94326ead-b2df-4e49-b1d7-357cc1d6d42f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:05.822297 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94326ead-b2df-4e49-b1d7-357cc1d6d42f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:05.822748 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94326ead-b2df-4e49-b1d7-357cc1d6d42f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:05.823282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94326ead-b2df-4e49-b1d7-357cc1d6d42f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:05.900864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-94326ead-b2df-4e49-b1d7-357cc1d6d42f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:05.912708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-94326ead-b2df-4e49-b1d7-357cc1d6d42f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.104589 Aug 07 01:31:05.913314 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 888/1602] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:05 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:06.935087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8686e007-ce12-4140-a96e-15e8758b1f00 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:06.935733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8686e007-ce12-4140-a96e-15e8758b1f00 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:06.943592 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8686e007-ce12-4140-a96e-15e8758b1f00 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:06.944091 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8686e007-ce12-4140-a96e-15e8758b1f00 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:06.944568 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8686e007-ce12-4140-a96e-15e8758b1f00 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:06.945315 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8686e007-ce12-4140-a96e-15e8758b1f00 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:06.945796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8686e007-ce12-4140-a96e-15e8758b1f00 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:06.946257 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8686e007-ce12-4140-a96e-15e8758b1f00 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:07.028644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8686e007-ce12-4140-a96e-15e8758b1f00 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:07.038084 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8686e007-ce12-4140-a96e-15e8758b1f00 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.106911 Aug 07 01:31:07.038722 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 715/1603] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:06 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:08.071703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bb4ae386-772c-415a-a9df-fa6bd92b48b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:08.072593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bb4ae386-772c-415a-a9df-fa6bd92b48b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:08.081272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb4ae386-772c-415a-a9df-fa6bd92b48b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:08.081779 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb4ae386-772c-415a-a9df-fa6bd92b48b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:08.082230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb4ae386-772c-415a-a9df-fa6bd92b48b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:08.082978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb4ae386-772c-415a-a9df-fa6bd92b48b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:08.083437 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb4ae386-772c-415a-a9df-fa6bd92b48b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:08.083981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb4ae386-772c-415a-a9df-fa6bd92b48b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:08.156898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-bb4ae386-772c-415a-a9df-fa6bd92b48b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:08.171348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bb4ae386-772c-415a-a9df-fa6bd92b48b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.104301 Aug 07 01:31:08.172380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 889/1604] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:08 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:09.193856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5abaf091-b5db-4811-a845-dd18005eb3f2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:09.194524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5abaf091-b5db-4811-a845-dd18005eb3f2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:09.201841 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5abaf091-b5db-4811-a845-dd18005eb3f2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:09.202379 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5abaf091-b5db-4811-a845-dd18005eb3f2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:09.202884 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5abaf091-b5db-4811-a845-dd18005eb3f2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:09.204147 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5abaf091-b5db-4811-a845-dd18005eb3f2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:09.204703 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5abaf091-b5db-4811-a845-dd18005eb3f2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:09.205183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5abaf091-b5db-4811-a845-dd18005eb3f2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:09.278649 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5abaf091-b5db-4811-a845-dd18005eb3f2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:09.293639 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5abaf091-b5db-4811-a845-dd18005eb3f2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.103677 Aug 07 01:31:09.294693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 716/1605] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:09 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:10.317603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-94d8a884-0653-454c-a598-1114c821c9af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:10.318234 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-94d8a884-0653-454c-a598-1114c821c9af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:10.325341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94d8a884-0653-454c-a598-1114c821c9af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:10.325828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94d8a884-0653-454c-a598-1114c821c9af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:10.326262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94d8a884-0653-454c-a598-1114c821c9af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:10.327361 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94d8a884-0653-454c-a598-1114c821c9af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:10.328467 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94d8a884-0653-454c-a598-1114c821c9af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:10.329202 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94d8a884-0653-454c-a598-1114c821c9af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:10.404005 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-94d8a884-0653-454c-a598-1114c821c9af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:10.419396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-94d8a884-0653-454c-a598-1114c821c9af tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.106082 Aug 07 01:31:10.420457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 890/1606] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:10 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:11.453747 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-facc13a1-7712-4be7-a113-42bc97b9d7e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:11.454530 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-facc13a1-7712-4be7-a113-42bc97b9d7e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:11.461950 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-facc13a1-7712-4be7-a113-42bc97b9d7e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:11.462587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-facc13a1-7712-4be7-a113-42bc97b9d7e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:11.463241 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-facc13a1-7712-4be7-a113-42bc97b9d7e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:11.464113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-facc13a1-7712-4be7-a113-42bc97b9d7e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:11.464707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-facc13a1-7712-4be7-a113-42bc97b9d7e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:11.465235 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-facc13a1-7712-4be7-a113-42bc97b9d7e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:11.534301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-facc13a1-7712-4be7-a113-42bc97b9d7e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:11.542843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-facc13a1-7712-4be7-a113-42bc97b9d7e7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.093225 Aug 07 01:31:11.543496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 717/1607] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:11 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:12.566523 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f32e4418-fe95-4c81-961e-947a498c7e05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:12.567435 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f32e4418-fe95-4c81-961e-947a498c7e05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:12.575600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f32e4418-fe95-4c81-961e-947a498c7e05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:12.576160 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f32e4418-fe95-4c81-961e-947a498c7e05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:12.576745 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f32e4418-fe95-4c81-961e-947a498c7e05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:12.577576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f32e4418-fe95-4c81-961e-947a498c7e05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:12.578417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f32e4418-fe95-4c81-961e-947a498c7e05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:12.579502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f32e4418-fe95-4c81-961e-947a498c7e05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:12.658597 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f32e4418-fe95-4c81-961e-947a498c7e05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:12.667939 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f32e4418-fe95-4c81-961e-947a498c7e05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.105795 Aug 07 01:31:12.668722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 891/1608] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:12 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:13.690175 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e9e207a9-733a-4fb4-89ca-c249123968d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:13.690781 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e9e207a9-733a-4fb4-89ca-c249123968d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:13.698043 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e9e207a9-733a-4fb4-89ca-c249123968d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:13.698538 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e9e207a9-733a-4fb4-89ca-c249123968d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:13.698973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e9e207a9-733a-4fb4-89ca-c249123968d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:13.699690 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e9e207a9-733a-4fb4-89ca-c249123968d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:13.700150 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e9e207a9-733a-4fb4-89ca-c249123968d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:13.700648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e9e207a9-733a-4fb4-89ca-c249123968d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:13.769856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e9e207a9-733a-4fb4-89ca-c249123968d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:13.779038 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e9e207a9-733a-4fb4-89ca-c249123968d3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.093251 Aug 07 01:31:13.779715 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 718/1609] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:13 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:14.802167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e5a96760-043e-4d96-aabe-14c42a74ebb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:14.803053 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e5a96760-043e-4d96-aabe-14c42a74ebb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:14.810715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5a96760-043e-4d96-aabe-14c42a74ebb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:14.811302 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5a96760-043e-4d96-aabe-14c42a74ebb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:14.811806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5a96760-043e-4d96-aabe-14c42a74ebb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:14.812625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5a96760-043e-4d96-aabe-14c42a74ebb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:14.813232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5a96760-043e-4d96-aabe-14c42a74ebb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:14.813707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e5a96760-043e-4d96-aabe-14c42a74ebb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:14.903336 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e5a96760-043e-4d96-aabe-14c42a74ebb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:14.918417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e5a96760-043e-4d96-aabe-14c42a74ebb1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.120366 Aug 07 01:31:14.920341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 892/1610] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:14 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:15.944061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1e6f3686-9f78-4304-bdf7-8f24fb6d7f01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:15.945210 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1e6f3686-9f78-4304-bdf7-8f24fb6d7f01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:15.953593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e6f3686-9f78-4304-bdf7-8f24fb6d7f01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:15.954095 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e6f3686-9f78-4304-bdf7-8f24fb6d7f01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:15.954540 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e6f3686-9f78-4304-bdf7-8f24fb6d7f01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:15.955362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e6f3686-9f78-4304-bdf7-8f24fb6d7f01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:15.955827 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e6f3686-9f78-4304-bdf7-8f24fb6d7f01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:15.956279 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e6f3686-9f78-4304-bdf7-8f24fb6d7f01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:16.029307 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1e6f3686-9f78-4304-bdf7-8f24fb6d7f01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:16.039093 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1e6f3686-9f78-4304-bdf7-8f24fb6d7f01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.099168 Aug 07 01:31:16.039771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 719/1611] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:15 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:17.074870 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ca5ed0a5-9d25-4af5-a210-35d413230c6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:17.075508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ca5ed0a5-9d25-4af5-a210-35d413230c6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:17.083681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ca5ed0a5-9d25-4af5-a210-35d413230c6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:17.084175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ca5ed0a5-9d25-4af5-a210-35d413230c6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:17.084719 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ca5ed0a5-9d25-4af5-a210-35d413230c6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:17.085651 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ca5ed0a5-9d25-4af5-a210-35d413230c6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:17.086156 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ca5ed0a5-9d25-4af5-a210-35d413230c6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:17.086728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ca5ed0a5-9d25-4af5-a210-35d413230c6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:17.167213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ca5ed0a5-9d25-4af5-a210-35d413230c6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:17.178275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ca5ed0a5-9d25-4af5-a210-35d413230c6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.107648 Aug 07 01:31:17.179209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 893/1612] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:17 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:18.200491 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-06e8c1f7-9f42-45a7-92aa-63e49445f669 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:18.201391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-06e8c1f7-9f42-45a7-92aa-63e49445f669 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:18.208442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06e8c1f7-9f42-45a7-92aa-63e49445f669 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:18.208932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06e8c1f7-9f42-45a7-92aa-63e49445f669 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:18.209370 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06e8c1f7-9f42-45a7-92aa-63e49445f669 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:18.210086 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06e8c1f7-9f42-45a7-92aa-63e49445f669 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:18.210539 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06e8c1f7-9f42-45a7-92aa-63e49445f669 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:18.210960 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06e8c1f7-9f42-45a7-92aa-63e49445f669 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:18.284013 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-06e8c1f7-9f42-45a7-92aa-63e49445f669 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:18.292891 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-06e8c1f7-9f42-45a7-92aa-63e49445f669 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.096134 Aug 07 01:31:18.293579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 720/1613] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:18 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:19.316642 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6ba42d7f-10dd-4b4e-bc7c-cc4d31bef3ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:19.325194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6ba42d7f-10dd-4b4e-bc7c-cc4d31bef3ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:19.333788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba42d7f-10dd-4b4e-bc7c-cc4d31bef3ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:19.334295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba42d7f-10dd-4b4e-bc7c-cc4d31bef3ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:19.334738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba42d7f-10dd-4b4e-bc7c-cc4d31bef3ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:19.335672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba42d7f-10dd-4b4e-bc7c-cc4d31bef3ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:19.336542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba42d7f-10dd-4b4e-bc7c-cc4d31bef3ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:19.337260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ba42d7f-10dd-4b4e-bc7c-cc4d31bef3ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:19.423449 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6ba42d7f-10dd-4b4e-bc7c-cc4d31bef3ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:19.433533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6ba42d7f-10dd-4b4e-bc7c-cc4d31bef3ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.122119 Aug 07 01:31:19.434130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 894/1614] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:19 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:20.458347 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b9db147b-5d95-4bcc-997f-f186c9305f7f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:20.459046 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b9db147b-5d95-4bcc-997f-f186c9305f7f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:20.465764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b9db147b-5d95-4bcc-997f-f186c9305f7f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:20.466260 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b9db147b-5d95-4bcc-997f-f186c9305f7f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:20.466704 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b9db147b-5d95-4bcc-997f-f186c9305f7f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:20.467437 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b9db147b-5d95-4bcc-997f-f186c9305f7f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:20.467892 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b9db147b-5d95-4bcc-997f-f186c9305f7f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:20.468349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b9db147b-5d95-4bcc-997f-f186c9305f7f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:20.546785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b9db147b-5d95-4bcc-997f-f186c9305f7f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:20.560507 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b9db147b-5d95-4bcc-997f-f186c9305f7f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.105903 Aug 07 01:31:20.561403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 721/1615] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:20 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:21.595776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d04aadd2-e188-42a6-928a-612054663467 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:21.596676 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d04aadd2-e188-42a6-928a-612054663467 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:21.603744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d04aadd2-e188-42a6-928a-612054663467 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:21.604253 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d04aadd2-e188-42a6-928a-612054663467 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:21.604710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d04aadd2-e188-42a6-928a-612054663467 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:21.605440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d04aadd2-e188-42a6-928a-612054663467 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:21.605901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d04aadd2-e188-42a6-928a-612054663467 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:21.606328 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d04aadd2-e188-42a6-928a-612054663467 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:21.609446 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=80 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:31:21.692670 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d04aadd2-e188-42a6-928a-612054663467 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:21.703934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d04aadd2-e188-42a6-928a-612054663467 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.113408 Aug 07 01:31:21.704741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 895/1616] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:21 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:22.736546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-227c4bc2-d9be-4e9b-b498-88e198d27903 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:22.737205 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-227c4bc2-d9be-4e9b-b498-88e198d27903 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:22.745055 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-227c4bc2-d9be-4e9b-b498-88e198d27903 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:22.745545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-227c4bc2-d9be-4e9b-b498-88e198d27903 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:22.745995 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-227c4bc2-d9be-4e9b-b498-88e198d27903 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:22.746725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-227c4bc2-d9be-4e9b-b498-88e198d27903 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:22.747203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-227c4bc2-d9be-4e9b-b498-88e198d27903 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:22.747614 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-227c4bc2-d9be-4e9b-b498-88e198d27903 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:22.818906 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-227c4bc2-d9be-4e9b-b498-88e198d27903 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:22.829186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-227c4bc2-d9be-4e9b-b498-88e198d27903 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097846 Aug 07 01:31:22.829871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 722/1617] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:22 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:23.853272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-320ffc84-96f9-46eb-92a9-152ac3070a3f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:23.854611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-320ffc84-96f9-46eb-92a9-152ac3070a3f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:23.860456 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=28 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:31:23.866700 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-320ffc84-96f9-46eb-92a9-152ac3070a3f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:23.867377 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-320ffc84-96f9-46eb-92a9-152ac3070a3f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:23.868137 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-320ffc84-96f9-46eb-92a9-152ac3070a3f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:23.869264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-320ffc84-96f9-46eb-92a9-152ac3070a3f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:23.869732 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-320ffc84-96f9-46eb-92a9-152ac3070a3f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:23.870153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-320ffc84-96f9-46eb-92a9-152ac3070a3f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:23.961175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-320ffc84-96f9-46eb-92a9-152ac3070a3f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:23.973717 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-320ffc84-96f9-46eb-92a9-152ac3070a3f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.124899 Aug 07 01:31:23.974586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 896/1618] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:23 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:24.995751 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0810e3ed-4dcd-44fc-8d81-1b29c3c790ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:24.996395 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0810e3ed-4dcd-44fc-8d81-1b29c3c790ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:24.998907 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=28 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:31:25.005110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0810e3ed-4dcd-44fc-8d81-1b29c3c790ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:25.005612 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0810e3ed-4dcd-44fc-8d81-1b29c3c790ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:25.006050 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0810e3ed-4dcd-44fc-8d81-1b29c3c790ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:25.006777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0810e3ed-4dcd-44fc-8d81-1b29c3c790ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:25.007232 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0810e3ed-4dcd-44fc-8d81-1b29c3c790ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:25.007662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0810e3ed-4dcd-44fc-8d81-1b29c3c790ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:25.011131 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=84 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:31:25.089072 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0810e3ed-4dcd-44fc-8d81-1b29c3c790ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:25.097991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0810e3ed-4dcd-44fc-8d81-1b29c3c790ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.106131 Aug 07 01:31:25.098586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 723/1619] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:24 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:26.120399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-61010b63-3b5f-4762-9f74-019a6ee87825 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:26.121239 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-61010b63-3b5f-4762-9f74-019a6ee87825 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:26.129458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61010b63-3b5f-4762-9f74-019a6ee87825 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:26.130010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61010b63-3b5f-4762-9f74-019a6ee87825 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:26.130492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61010b63-3b5f-4762-9f74-019a6ee87825 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:26.131412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61010b63-3b5f-4762-9f74-019a6ee87825 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:26.131918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61010b63-3b5f-4762-9f74-019a6ee87825 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:26.132506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-61010b63-3b5f-4762-9f74-019a6ee87825 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:26.205355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-61010b63-3b5f-4762-9f74-019a6ee87825 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:26.220541 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-61010b63-3b5f-4762-9f74-019a6ee87825 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.104100 Aug 07 01:31:26.221662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 897/1620] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:26 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:27.243365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-371ff6b0-ea54-4ba7-ada8-6b645cc99df9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:27.244021 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-371ff6b0-ea54-4ba7-ada8-6b645cc99df9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:27.251340 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-371ff6b0-ea54-4ba7-ada8-6b645cc99df9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:27.251838 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-371ff6b0-ea54-4ba7-ada8-6b645cc99df9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:27.252307 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-371ff6b0-ea54-4ba7-ada8-6b645cc99df9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:27.253029 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-371ff6b0-ea54-4ba7-ada8-6b645cc99df9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:27.253485 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-371ff6b0-ea54-4ba7-ada8-6b645cc99df9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:27.253908 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-371ff6b0-ea54-4ba7-ada8-6b645cc99df9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:27.334857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-371ff6b0-ea54-4ba7-ada8-6b645cc99df9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:27.344910 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-371ff6b0-ea54-4ba7-ada8-6b645cc99df9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.105329 Aug 07 01:31:27.345515 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 724/1621] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:27 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:28.367191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a12f6da4-0bde-465d-bb4f-2c109c292140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:28.367982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a12f6da4-0bde-465d-bb4f-2c109c292140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:28.375268 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a12f6da4-0bde-465d-bb4f-2c109c292140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:28.376373 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a12f6da4-0bde-465d-bb4f-2c109c292140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:28.376811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a12f6da4-0bde-465d-bb4f-2c109c292140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:28.377527 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a12f6da4-0bde-465d-bb4f-2c109c292140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:28.377980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a12f6da4-0bde-465d-bb4f-2c109c292140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:28.378401 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a12f6da4-0bde-465d-bb4f-2c109c292140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:28.456473 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a12f6da4-0bde-465d-bb4f-2c109c292140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:28.472522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a12f6da4-0bde-465d-bb4f-2c109c292140 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.109107 Aug 07 01:31:28.473588 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 898/1622] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:28 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:29.495773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-dc66ae71-fe40-4700-8aef-be50de91d6c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:29.496460 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-dc66ae71-fe40-4700-8aef-be50de91d6c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:29.503524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc66ae71-fe40-4700-8aef-be50de91d6c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:29.504010 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc66ae71-fe40-4700-8aef-be50de91d6c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:29.504554 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc66ae71-fe40-4700-8aef-be50de91d6c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:29.505353 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc66ae71-fe40-4700-8aef-be50de91d6c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:29.505809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc66ae71-fe40-4700-8aef-be50de91d6c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:29.506230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc66ae71-fe40-4700-8aef-be50de91d6c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:29.577216 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dc66ae71-fe40-4700-8aef-be50de91d6c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:29.588824 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-dc66ae71-fe40-4700-8aef-be50de91d6c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.096933 Aug 07 01:31:29.589699 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 725/1623] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:29 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:30.612048 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1e8c0157-b74e-40ec-b413-6b4fac089b01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:30.612775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1e8c0157-b74e-40ec-b413-6b4fac089b01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:30.619844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e8c0157-b74e-40ec-b413-6b4fac089b01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:30.620368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e8c0157-b74e-40ec-b413-6b4fac089b01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:30.620818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e8c0157-b74e-40ec-b413-6b4fac089b01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:30.621555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e8c0157-b74e-40ec-b413-6b4fac089b01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:30.622023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e8c0157-b74e-40ec-b413-6b4fac089b01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:30.622453 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1e8c0157-b74e-40ec-b413-6b4fac089b01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:30.695328 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1e8c0157-b74e-40ec-b413-6b4fac089b01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:30.705096 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1e8c0157-b74e-40ec-b413-6b4fac089b01 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097114 Aug 07 01:31:30.705710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 899/1624] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:30 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:31.729099 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f16c470d-7d3b-45c5-bebf-0b719c497bb4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:31.729814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f16c470d-7d3b-45c5-bebf-0b719c497bb4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:31.738464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f16c470d-7d3b-45c5-bebf-0b719c497bb4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:31.739193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f16c470d-7d3b-45c5-bebf-0b719c497bb4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:31.739879 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f16c470d-7d3b-45c5-bebf-0b719c497bb4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:31.741075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f16c470d-7d3b-45c5-bebf-0b719c497bb4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:31.741772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f16c470d-7d3b-45c5-bebf-0b719c497bb4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:31.742390 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f16c470d-7d3b-45c5-bebf-0b719c497bb4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:31.832288 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f16c470d-7d3b-45c5-bebf-0b719c497bb4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:31.841994 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f16c470d-7d3b-45c5-bebf-0b719c497bb4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.119249 Aug 07 01:31:31.842604 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 726/1625] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:31 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:32.864535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cd393e81-0f75-468d-b784-c439667f8ad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:32.865311 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cd393e81-0f75-468d-b784-c439667f8ad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:32.872395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd393e81-0f75-468d-b784-c439667f8ad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:32.872895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd393e81-0f75-468d-b784-c439667f8ad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:32.873341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd393e81-0f75-468d-b784-c439667f8ad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:32.874086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd393e81-0f75-468d-b784-c439667f8ad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:32.874551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd393e81-0f75-468d-b784-c439667f8ad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:32.874987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cd393e81-0f75-468d-b784-c439667f8ad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:32.949157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-cd393e81-0f75-468d-b784-c439667f8ad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:32.959833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cd393e81-0f75-468d-b784-c439667f8ad6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.099338 Aug 07 01:31:32.960540 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 900/1626] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:32 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:33.996531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-954ee0b2-2c78-41e3-b5f4-85ec7c35dd16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:33.997728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-954ee0b2-2c78-41e3-b5f4-85ec7c35dd16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:34.009679 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-954ee0b2-2c78-41e3-b5f4-85ec7c35dd16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:34.010511 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-954ee0b2-2c78-41e3-b5f4-85ec7c35dd16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:34.011279 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-954ee0b2-2c78-41e3-b5f4-85ec7c35dd16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:34.012581 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-954ee0b2-2c78-41e3-b5f4-85ec7c35dd16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:34.013414 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-954ee0b2-2c78-41e3-b5f4-85ec7c35dd16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:34.014175 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-954ee0b2-2c78-41e3-b5f4-85ec7c35dd16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:34.093246 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-954ee0b2-2c78-41e3-b5f4-85ec7c35dd16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:34.103033 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-954ee0b2-2c78-41e3-b5f4-85ec7c35dd16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.114092 Aug 07 01:31:34.103623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 727/1627] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:33 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:35.126797 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0c8107f5-0623-4e32-8e65-3caa274c0063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:35.127521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0c8107f5-0623-4e32-8e65-3caa274c0063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:35.134838 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c8107f5-0623-4e32-8e65-3caa274c0063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:35.135379 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c8107f5-0623-4e32-8e65-3caa274c0063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:35.136051 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c8107f5-0623-4e32-8e65-3caa274c0063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:35.137356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c8107f5-0623-4e32-8e65-3caa274c0063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:35.138003 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c8107f5-0623-4e32-8e65-3caa274c0063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:35.138444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c8107f5-0623-4e32-8e65-3caa274c0063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:35.230100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0c8107f5-0623-4e32-8e65-3caa274c0063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:35.239213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0c8107f5-0623-4e32-8e65-3caa274c0063 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.117146 Aug 07 01:31:35.239964 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 901/1628] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:35 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:36.275256 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-00d1bc73-b8df-4c20-be37-74204b6db755 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:36.275911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-00d1bc73-b8df-4c20-be37-74204b6db755 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:36.283857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00d1bc73-b8df-4c20-be37-74204b6db755 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:36.284604 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00d1bc73-b8df-4c20-be37-74204b6db755 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:36.284856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00d1bc73-b8df-4c20-be37-74204b6db755 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:36.285660 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00d1bc73-b8df-4c20-be37-74204b6db755 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:36.286007 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00d1bc73-b8df-4c20-be37-74204b6db755 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:36.286437 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00d1bc73-b8df-4c20-be37-74204b6db755 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:36.529676 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-00d1bc73-b8df-4c20-be37-74204b6db755 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:36.539705 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-00d1bc73-b8df-4c20-be37-74204b6db755 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.268699 Aug 07 01:31:36.540466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 728/1629] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:36 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:37.562467 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-51dad64a-915c-4638-bc8a-24408c3786c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:37.563815 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-51dad64a-915c-4638-bc8a-24408c3786c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:37.570960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51dad64a-915c-4638-bc8a-24408c3786c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:37.571460 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51dad64a-915c-4638-bc8a-24408c3786c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:37.571902 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51dad64a-915c-4638-bc8a-24408c3786c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:37.572666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51dad64a-915c-4638-bc8a-24408c3786c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:37.573130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51dad64a-915c-4638-bc8a-24408c3786c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:37.573559 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51dad64a-915c-4638-bc8a-24408c3786c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:37.646741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-51dad64a-915c-4638-bc8a-24408c3786c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:37.660350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-51dad64a-915c-4638-bc8a-24408c3786c7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.101647 Aug 07 01:31:37.661402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 902/1630] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:37 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:38.685821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-06eefce7-3292-4b89-bc4d-534682572453 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:38.686565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-06eefce7-3292-4b89-bc4d-534682572453 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:38.694050 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06eefce7-3292-4b89-bc4d-534682572453 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:38.694601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06eefce7-3292-4b89-bc4d-534682572453 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:38.695092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06eefce7-3292-4b89-bc4d-534682572453 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:38.695928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06eefce7-3292-4b89-bc4d-534682572453 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:38.696535 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06eefce7-3292-4b89-bc4d-534682572453 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:38.697111 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-06eefce7-3292-4b89-bc4d-534682572453 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:38.768017 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-06eefce7-3292-4b89-bc4d-534682572453 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:38.778076 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-06eefce7-3292-4b89-bc4d-534682572453 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.096785 Aug 07 01:31:38.778937 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 729/1631] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:38 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:39.801306 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7c838eaf-e07e-4934-962f-62159ee8b942 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:39.802003 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7c838eaf-e07e-4934-962f-62159ee8b942 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:39.809465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c838eaf-e07e-4934-962f-62159ee8b942 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:39.809961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c838eaf-e07e-4934-962f-62159ee8b942 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:39.810408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c838eaf-e07e-4934-962f-62159ee8b942 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:39.811142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c838eaf-e07e-4934-962f-62159ee8b942 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:39.811600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c838eaf-e07e-4934-962f-62159ee8b942 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:39.812028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7c838eaf-e07e-4934-962f-62159ee8b942 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:39.883108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7c838eaf-e07e-4934-962f-62159ee8b942 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:39.893103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7c838eaf-e07e-4934-962f-62159ee8b942 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.096203 Aug 07 01:31:39.893753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 903/1632] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:39 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:40.922988 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-743ead5d-0815-4aaf-a7ec-cc86e6d6be27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:40.924197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-743ead5d-0815-4aaf-a7ec-cc86e6d6be27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:40.935656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-743ead5d-0815-4aaf-a7ec-cc86e6d6be27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:40.936466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-743ead5d-0815-4aaf-a7ec-cc86e6d6be27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:40.937317 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-743ead5d-0815-4aaf-a7ec-cc86e6d6be27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:40.938592 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-743ead5d-0815-4aaf-a7ec-cc86e6d6be27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:40.939387 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-743ead5d-0815-4aaf-a7ec-cc86e6d6be27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:40.940116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-743ead5d-0815-4aaf-a7ec-cc86e6d6be27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:41.018632 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-743ead5d-0815-4aaf-a7ec-cc86e6d6be27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:41.030247 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-743ead5d-0815-4aaf-a7ec-cc86e6d6be27 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.114675 Aug 07 01:31:41.031282 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 730/1633] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:40 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:42.052906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a9707b43-0086-4559-b8d2-b288f5ad5f19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:42.053725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a9707b43-0086-4559-b8d2-b288f5ad5f19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:42.060657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9707b43-0086-4559-b8d2-b288f5ad5f19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:42.061234 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9707b43-0086-4559-b8d2-b288f5ad5f19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:42.061630 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9707b43-0086-4559-b8d2-b288f5ad5f19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:42.062453 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9707b43-0086-4559-b8d2-b288f5ad5f19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:42.063071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9707b43-0086-4559-b8d2-b288f5ad5f19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:42.063516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9707b43-0086-4559-b8d2-b288f5ad5f19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:42.137598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a9707b43-0086-4559-b8d2-b288f5ad5f19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:42.149916 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a9707b43-0086-4559-b8d2-b288f5ad5f19 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.100843 Aug 07 01:31:42.150529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 904/1634] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:42 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:43.186743 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f264385e-7897-4d58-8124-8d517cafe53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:43.188035 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f264385e-7897-4d58-8124-8d517cafe53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:43.195577 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f264385e-7897-4d58-8124-8d517cafe53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:43.196100 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f264385e-7897-4d58-8124-8d517cafe53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:43.196588 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f264385e-7897-4d58-8124-8d517cafe53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:43.197417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f264385e-7897-4d58-8124-8d517cafe53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:43.197881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f264385e-7897-4d58-8124-8d517cafe53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:43.198320 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f264385e-7897-4d58-8124-8d517cafe53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:43.272420 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f264385e-7897-4d58-8124-8d517cafe53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:43.284451 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f264385e-7897-4d58-8124-8d517cafe53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.104816 Aug 07 01:31:43.285282 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 731/1635] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:43 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:44.309564 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dfeb1e64-6957-4cec-9981-db4bad9697bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:44.310316 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dfeb1e64-6957-4cec-9981-db4bad9697bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:44.318487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dfeb1e64-6957-4cec-9981-db4bad9697bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:44.319104 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dfeb1e64-6957-4cec-9981-db4bad9697bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:44.319567 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dfeb1e64-6957-4cec-9981-db4bad9697bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:44.320565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dfeb1e64-6957-4cec-9981-db4bad9697bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:44.321132 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dfeb1e64-6957-4cec-9981-db4bad9697bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:44.321664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dfeb1e64-6957-4cec-9981-db4bad9697bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:44.391536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-dfeb1e64-6957-4cec-9981-db4bad9697bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:44.400660 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dfeb1e64-6957-4cec-9981-db4bad9697bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097193 Aug 07 01:31:44.401270 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 905/1636] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:44 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:45.434301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-631cce19-2622-40bf-98a7-79744340f156 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:45.434975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-631cce19-2622-40bf-98a7-79744340f156 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:45.442451 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-631cce19-2622-40bf-98a7-79744340f156 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:45.442936 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-631cce19-2622-40bf-98a7-79744340f156 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:45.443422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-631cce19-2622-40bf-98a7-79744340f156 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:45.444217 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-631cce19-2622-40bf-98a7-79744340f156 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:45.444790 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-631cce19-2622-40bf-98a7-79744340f156 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:45.445368 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-631cce19-2622-40bf-98a7-79744340f156 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:45.537465 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-631cce19-2622-40bf-98a7-79744340f156 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:45.546045 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-631cce19-2622-40bf-98a7-79744340f156 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.115717 Aug 07 01:31:45.546642 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 732/1637] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:45 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:46.569741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1fa70e51-5555-4509-aa14-dea661bb2eb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:46.570657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1fa70e51-5555-4509-aa14-dea661bb2eb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:46.577966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1fa70e51-5555-4509-aa14-dea661bb2eb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:46.579252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1fa70e51-5555-4509-aa14-dea661bb2eb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:46.579781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1fa70e51-5555-4509-aa14-dea661bb2eb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:46.580601 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1fa70e51-5555-4509-aa14-dea661bb2eb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:46.581083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1fa70e51-5555-4509-aa14-dea661bb2eb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:46.581519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1fa70e51-5555-4509-aa14-dea661bb2eb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:46.658411 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1fa70e51-5555-4509-aa14-dea661bb2eb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:46.674875 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1fa70e51-5555-4509-aa14-dea661bb2eb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.109267 Aug 07 01:31:46.675959 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 906/1638] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:46 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:47.696834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-77c9a12f-fe3e-4349-9497-127b1bdd00d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:47.697425 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-77c9a12f-fe3e-4349-9497-127b1bdd00d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:47.706454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77c9a12f-fe3e-4349-9497-127b1bdd00d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:47.707004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77c9a12f-fe3e-4349-9497-127b1bdd00d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:47.707482 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77c9a12f-fe3e-4349-9497-127b1bdd00d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:47.708334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77c9a12f-fe3e-4349-9497-127b1bdd00d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:47.708939 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77c9a12f-fe3e-4349-9497-127b1bdd00d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:47.709422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-77c9a12f-fe3e-4349-9497-127b1bdd00d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:47.782932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-77c9a12f-fe3e-4349-9497-127b1bdd00d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:47.792442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-77c9a12f-fe3e-4349-9497-127b1bdd00d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.099863 Aug 07 01:31:47.793056 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 733/1639] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:47 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:48.818170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2b1543ed-dd16-4842-a716-d77fc2a4bbb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:48.819356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2b1543ed-dd16-4842-a716-d77fc2a4bbb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:48.831281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b1543ed-dd16-4842-a716-d77fc2a4bbb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:48.832043 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b1543ed-dd16-4842-a716-d77fc2a4bbb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:48.832751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b1543ed-dd16-4842-a716-d77fc2a4bbb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:48.833835 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b1543ed-dd16-4842-a716-d77fc2a4bbb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:48.834485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b1543ed-dd16-4842-a716-d77fc2a4bbb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:48.835149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2b1543ed-dd16-4842-a716-d77fc2a4bbb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:48.915711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2b1543ed-dd16-4842-a716-d77fc2a4bbb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:48.926052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2b1543ed-dd16-4842-a716-d77fc2a4bbb0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.115494 Aug 07 01:31:48.926697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 907/1640] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:48 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:49.952354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0e5cdb4c-be8d-4735-9ebe-1ff45844a7ba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:49.954058 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0e5cdb4c-be8d-4735-9ebe-1ff45844a7ba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:49.967343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e5cdb4c-be8d-4735-9ebe-1ff45844a7ba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:49.968457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e5cdb4c-be8d-4735-9ebe-1ff45844a7ba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:49.969839 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e5cdb4c-be8d-4735-9ebe-1ff45844a7ba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:49.971277 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e5cdb4c-be8d-4735-9ebe-1ff45844a7ba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:49.972152 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e5cdb4c-be8d-4735-9ebe-1ff45844a7ba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:49.972991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0e5cdb4c-be8d-4735-9ebe-1ff45844a7ba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:50.054625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0e5cdb4c-be8d-4735-9ebe-1ff45844a7ba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:50.063710 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0e5cdb4c-be8d-4735-9ebe-1ff45844a7ba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.118991 Aug 07 01:31:50.064404 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 734/1641] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:49 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:51.086614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-17a66ec0-0918-4721-88c8-dec9288f596c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:51.087399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-17a66ec0-0918-4721-88c8-dec9288f596c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:51.094322 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17a66ec0-0918-4721-88c8-dec9288f596c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:51.094863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17a66ec0-0918-4721-88c8-dec9288f596c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:51.095350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17a66ec0-0918-4721-88c8-dec9288f596c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:51.096322 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17a66ec0-0918-4721-88c8-dec9288f596c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:51.096772 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17a66ec0-0918-4721-88c8-dec9288f596c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:51.097239 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17a66ec0-0918-4721-88c8-dec9288f596c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:51.189531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-17a66ec0-0918-4721-88c8-dec9288f596c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:51.198665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-17a66ec0-0918-4721-88c8-dec9288f596c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.116503 Aug 07 01:31:51.199268 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 908/1642] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:51 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:52.223336 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d240f731-f264-4f48-8504-604b344c57d8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:52.223938 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d240f731-f264-4f48-8504-604b344c57d8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:52.231776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d240f731-f264-4f48-8504-604b344c57d8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:52.232426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d240f731-f264-4f48-8504-604b344c57d8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:52.233261 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d240f731-f264-4f48-8504-604b344c57d8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:52.234025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d240f731-f264-4f48-8504-604b344c57d8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:52.235134 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d240f731-f264-4f48-8504-604b344c57d8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:52.235596 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d240f731-f264-4f48-8504-604b344c57d8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:52.313457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d240f731-f264-4f48-8504-604b344c57d8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:52.328389 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d240f731-f264-4f48-8504-604b344c57d8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.108943 Aug 07 01:31:52.329694 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 735/1643] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:52 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:53.362350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-582ee0ed-d570-45a6-930c-19718c27fcc2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:53.362996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-582ee0ed-d570-45a6-930c-19718c27fcc2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:53.369858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-582ee0ed-d570-45a6-930c-19718c27fcc2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:53.370347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-582ee0ed-d570-45a6-930c-19718c27fcc2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:53.370781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-582ee0ed-d570-45a6-930c-19718c27fcc2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:53.371494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-582ee0ed-d570-45a6-930c-19718c27fcc2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:53.371945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-582ee0ed-d570-45a6-930c-19718c27fcc2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:53.372408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-582ee0ed-d570-45a6-930c-19718c27fcc2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:53.375513 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=84 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:31:53.445414 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-582ee0ed-d570-45a6-930c-19718c27fcc2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:53.457152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-582ee0ed-d570-45a6-930c-19718c27fcc2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.099310 Aug 07 01:31:53.457751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 909/1644] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:53 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:54.220097 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:31:54.220415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:31:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:31:54.479662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1b5668e3-298b-4b60-b459-cb19663acf11 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:54.480366 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1b5668e3-298b-4b60-b459-cb19663acf11 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:54.487672 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5668e3-298b-4b60-b459-cb19663acf11 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:54.488172 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5668e3-298b-4b60-b459-cb19663acf11 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:54.488643 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5668e3-298b-4b60-b459-cb19663acf11 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:54.489374 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5668e3-298b-4b60-b459-cb19663acf11 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:54.489927 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5668e3-298b-4b60-b459-cb19663acf11 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:54.490355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1b5668e3-298b-4b60-b459-cb19663acf11 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:54.562385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1b5668e3-298b-4b60-b459-cb19663acf11 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:54.572223 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1b5668e3-298b-4b60-b459-cb19663acf11 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.096778 Aug 07 01:31:54.572953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 736/1645] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:54 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:31:55.599642 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2fe03d41-fbe3-4976-a936-d0ba75940d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:55.600765 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2fe03d41-fbe3-4976-a936-d0ba75940d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:55.604925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=28 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:31:55.614966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fe03d41-fbe3-4976-a936-d0ba75940d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:55.615744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fe03d41-fbe3-4976-a936-d0ba75940d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:55.616513 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fe03d41-fbe3-4976-a936-d0ba75940d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:55.617671 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fe03d41-fbe3-4976-a936-d0ba75940d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:55.618407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fe03d41-fbe3-4976-a936-d0ba75940d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:55.619121 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fe03d41-fbe3-4976-a936-d0ba75940d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:55.725529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2fe03d41-fbe3-4976-a936-d0ba75940d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:55.734479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2fe03d41-fbe3-4976-a936-d0ba75940d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.142755 Aug 07 01:31:55.735181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 910/1646] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:55 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:56.757121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e864b974-fb6d-48a4-b703-3bc0a084ee47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:56.757914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e864b974-fb6d-48a4-b703-3bc0a084ee47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:56.760428 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=28 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:31:56.766496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e864b974-fb6d-48a4-b703-3bc0a084ee47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:56.766993 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e864b974-fb6d-48a4-b703-3bc0a084ee47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:56.767434 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e864b974-fb6d-48a4-b703-3bc0a084ee47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:56.768189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e864b974-fb6d-48a4-b703-3bc0a084ee47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:56.768704 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e864b974-fb6d-48a4-b703-3bc0a084ee47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:56.769132 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e864b974-fb6d-48a4-b703-3bc0a084ee47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:56.773005 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=84 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:31:56.842552 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e864b974-fb6d-48a4-b703-3bc0a084ee47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:56.851917 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e864b974-fb6d-48a4-b703-3bc0a084ee47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098974 Aug 07 01:31:56.852546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 737/1647] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:56 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:57.874077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a5211fa8-429f-4be2-bf72-1d224f05fcec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:57.874790 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a5211fa8-429f-4be2-bf72-1d224f05fcec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:57.882858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5211fa8-429f-4be2-bf72-1d224f05fcec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:57.883460 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5211fa8-429f-4be2-bf72-1d224f05fcec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:57.883952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5211fa8-429f-4be2-bf72-1d224f05fcec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:57.884799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5211fa8-429f-4be2-bf72-1d224f05fcec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:57.885339 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5211fa8-429f-4be2-bf72-1d224f05fcec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:57.885918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5211fa8-429f-4be2-bf72-1d224f05fcec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:57.960555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a5211fa8-429f-4be2-bf72-1d224f05fcec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:57.969968 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a5211fa8-429f-4be2-bf72-1d224f05fcec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.099782 Aug 07 01:31:57.970442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 911/1648] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:57 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:31:58.992258 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d1d6cbcb-551a-4e32-9a12-573bc2dc0716 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:31:58.992866 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d1d6cbcb-551a-4e32-9a12-573bc2dc0716 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:31:59.000805 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1d6cbcb-551a-4e32-9a12-573bc2dc0716 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:59.001524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1d6cbcb-551a-4e32-9a12-573bc2dc0716 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:59.002046 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1d6cbcb-551a-4e32-9a12-573bc2dc0716 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:59.002823 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1d6cbcb-551a-4e32-9a12-573bc2dc0716 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:31:59.003339 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1d6cbcb-551a-4e32-9a12-573bc2dc0716 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:31:59.003878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d1d6cbcb-551a-4e32-9a12-573bc2dc0716 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:31:59.094194 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d1d6cbcb-551a-4e32-9a12-573bc2dc0716 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:31:59.103229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d1d6cbcb-551a-4e32-9a12-573bc2dc0716 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.115245 Aug 07 01:31:59.104003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 738/1649] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:31:58 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:00.125786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-69aa18c4-446a-4b1f-889a-311cd13b7804 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:00.126429 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-69aa18c4-446a-4b1f-889a-311cd13b7804 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:00.133545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69aa18c4-446a-4b1f-889a-311cd13b7804 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:00.134023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69aa18c4-446a-4b1f-889a-311cd13b7804 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:00.134474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69aa18c4-446a-4b1f-889a-311cd13b7804 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:00.135209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69aa18c4-446a-4b1f-889a-311cd13b7804 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:00.135703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69aa18c4-446a-4b1f-889a-311cd13b7804 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:00.136157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69aa18c4-446a-4b1f-889a-311cd13b7804 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:00.213564 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-69aa18c4-446a-4b1f-889a-311cd13b7804 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:00.228189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-69aa18c4-446a-4b1f-889a-311cd13b7804 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.106480 Aug 07 01:32:00.229275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 912/1650] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:00 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:01.251025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-526840a6-8d3c-4413-ad1b-b88c1bff0ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:01.251731 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-526840a6-8d3c-4413-ad1b-b88c1bff0ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:01.259894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-526840a6-8d3c-4413-ad1b-b88c1bff0ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:01.260490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-526840a6-8d3c-4413-ad1b-b88c1bff0ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:01.261012 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-526840a6-8d3c-4413-ad1b-b88c1bff0ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:01.262103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-526840a6-8d3c-4413-ad1b-b88c1bff0ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:01.262741 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-526840a6-8d3c-4413-ad1b-b88c1bff0ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:01.263299 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-526840a6-8d3c-4413-ad1b-b88c1bff0ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:01.362101 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-526840a6-8d3c-4413-ad1b-b88c1bff0ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:01.371491 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-526840a6-8d3c-4413-ad1b-b88c1bff0ab1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.125155 Aug 07 01:32:01.372090 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 739/1651] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:01 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:02.393470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f742f506-acd7-45c7-bcc6-d05d0d44066d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:02.394343 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f742f506-acd7-45c7-bcc6-d05d0d44066d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:02.402010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f742f506-acd7-45c7-bcc6-d05d0d44066d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:02.402714 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f742f506-acd7-45c7-bcc6-d05d0d44066d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:02.403273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f742f506-acd7-45c7-bcc6-d05d0d44066d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:02.404339 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f742f506-acd7-45c7-bcc6-d05d0d44066d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:02.404960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f742f506-acd7-45c7-bcc6-d05d0d44066d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:02.405425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f742f506-acd7-45c7-bcc6-d05d0d44066d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:02.478608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f742f506-acd7-45c7-bcc6-d05d0d44066d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:02.487888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f742f506-acd7-45c7-bcc6-d05d0d44066d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098492 Aug 07 01:32:02.488609 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 913/1652] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:02 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:03.510367 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6f296247-d202-4fe8-8399-cfa55c9f2970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:03.511145 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6f296247-d202-4fe8-8399-cfa55c9f2970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:03.518808 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6f296247-d202-4fe8-8399-cfa55c9f2970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:03.519311 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6f296247-d202-4fe8-8399-cfa55c9f2970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:03.519752 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6f296247-d202-4fe8-8399-cfa55c9f2970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:03.520702 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6f296247-d202-4fe8-8399-cfa55c9f2970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:03.520969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6f296247-d202-4fe8-8399-cfa55c9f2970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:03.521382 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6f296247-d202-4fe8-8399-cfa55c9f2970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:03.592226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6f296247-d202-4fe8-8399-cfa55c9f2970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:03.601230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6f296247-d202-4fe8-8399-cfa55c9f2970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094935 Aug 07 01:32:03.601854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 740/1653] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:03 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:04.528055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:32:04.528941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:32:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:32:04.624469 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c294e4a6-0ed5-4fdb-a5dd-84575d1795fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:04.625478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c294e4a6-0ed5-4fdb-a5dd-84575d1795fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:04.632768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c294e4a6-0ed5-4fdb-a5dd-84575d1795fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:04.633260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c294e4a6-0ed5-4fdb-a5dd-84575d1795fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:04.633700 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c294e4a6-0ed5-4fdb-a5dd-84575d1795fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:04.634429 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c294e4a6-0ed5-4fdb-a5dd-84575d1795fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:04.634886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c294e4a6-0ed5-4fdb-a5dd-84575d1795fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:04.635323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c294e4a6-0ed5-4fdb-a5dd-84575d1795fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:04.714830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c294e4a6-0ed5-4fdb-a5dd-84575d1795fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:04.730399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c294e4a6-0ed5-4fdb-a5dd-84575d1795fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.109958 Aug 07 01:32:04.731296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 914/1654] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:04 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:05.754232 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fd7198e0-0535-4474-8976-b2d07c054ff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:05.754861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fd7198e0-0535-4474-8976-b2d07c054ff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:05.762845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd7198e0-0535-4474-8976-b2d07c054ff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:05.763381 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd7198e0-0535-4474-8976-b2d07c054ff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:05.763920 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd7198e0-0535-4474-8976-b2d07c054ff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:05.764687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd7198e0-0535-4474-8976-b2d07c054ff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:05.765157 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd7198e0-0535-4474-8976-b2d07c054ff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:05.765592 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd7198e0-0535-4474-8976-b2d07c054ff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:05.855903 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fd7198e0-0535-4474-8976-b2d07c054ff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:05.865669 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fd7198e0-0535-4474-8976-b2d07c054ff0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.116634 Aug 07 01:32:05.866294 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 741/1655] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:05 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:06.887571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-097beb74-df03-4a83-a2ba-767e51773b10 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:06.888265 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-097beb74-df03-4a83-a2ba-767e51773b10 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:06.897163 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-097beb74-df03-4a83-a2ba-767e51773b10 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:06.897665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-097beb74-df03-4a83-a2ba-767e51773b10 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:06.898120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-097beb74-df03-4a83-a2ba-767e51773b10 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:06.898845 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-097beb74-df03-4a83-a2ba-767e51773b10 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:06.899320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-097beb74-df03-4a83-a2ba-767e51773b10 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:06.899755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-097beb74-df03-4a83-a2ba-767e51773b10 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:06.974005 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-097beb74-df03-4a83-a2ba-767e51773b10 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:06.986287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-097beb74-df03-4a83-a2ba-767e51773b10 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.102530 Aug 07 01:32:06.987047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 915/1656] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:06 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:08.012631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-dbb6f9ad-7b6f-463a-996c-1fb20c88f1f9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:08.013863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-dbb6f9ad-7b6f-463a-996c-1fb20c88f1f9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:08.025749 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dbb6f9ad-7b6f-463a-996c-1fb20c88f1f9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:08.026481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dbb6f9ad-7b6f-463a-996c-1fb20c88f1f9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:08.027138 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dbb6f9ad-7b6f-463a-996c-1fb20c88f1f9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:08.028247 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dbb6f9ad-7b6f-463a-996c-1fb20c88f1f9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:08.028958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dbb6f9ad-7b6f-463a-996c-1fb20c88f1f9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:08.029605 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dbb6f9ad-7b6f-463a-996c-1fb20c88f1f9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:08.123873 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dbb6f9ad-7b6f-463a-996c-1fb20c88f1f9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:08.139672 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-dbb6f9ad-7b6f-463a-996c-1fb20c88f1f9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.134233 Aug 07 01:32:08.140766 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 742/1657] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:08 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:09.174365 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9074efa1-ab72-4b1b-b80c-67a1b1c5c122 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:09.175094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9074efa1-ab72-4b1b-b80c-67a1b1c5c122 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:09.182913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9074efa1-ab72-4b1b-b80c-67a1b1c5c122 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:09.183461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9074efa1-ab72-4b1b-b80c-67a1b1c5c122 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:09.184102 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9074efa1-ab72-4b1b-b80c-67a1b1c5c122 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:09.185249 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9074efa1-ab72-4b1b-b80c-67a1b1c5c122 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:09.185664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9074efa1-ab72-4b1b-b80c-67a1b1c5c122 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:09.186107 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9074efa1-ab72-4b1b-b80c-67a1b1c5c122 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:09.261284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9074efa1-ab72-4b1b-b80c-67a1b1c5c122 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:09.276871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9074efa1-ab72-4b1b-b80c-67a1b1c5c122 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.106697 Aug 07 01:32:09.277891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 916/1658] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:09 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:10.315989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-72ad3692-960d-4501-83dd-333ef1f2f4bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:10.317061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-72ad3692-960d-4501-83dd-333ef1f2f4bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:10.325941 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-72ad3692-960d-4501-83dd-333ef1f2f4bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:10.326668 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-72ad3692-960d-4501-83dd-333ef1f2f4bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:10.327313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-72ad3692-960d-4501-83dd-333ef1f2f4bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:10.328800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-72ad3692-960d-4501-83dd-333ef1f2f4bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:10.329710 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-72ad3692-960d-4501-83dd-333ef1f2f4bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:10.330291 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-72ad3692-960d-4501-83dd-333ef1f2f4bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:10.428979 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-72ad3692-960d-4501-83dd-333ef1f2f4bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:10.441497 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-72ad3692-960d-4501-83dd-333ef1f2f4bf tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.129949 Aug 07 01:32:10.442537 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 743/1659] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:10 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:11.466830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-650cbdbc-7fe0-40b6-8a1e-709f870e9eec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:11.467485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-650cbdbc-7fe0-40b6-8a1e-709f870e9eec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:11.474975 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-650cbdbc-7fe0-40b6-8a1e-709f870e9eec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:11.475469 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-650cbdbc-7fe0-40b6-8a1e-709f870e9eec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:11.475906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-650cbdbc-7fe0-40b6-8a1e-709f870e9eec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:11.476662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-650cbdbc-7fe0-40b6-8a1e-709f870e9eec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:11.477109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-650cbdbc-7fe0-40b6-8a1e-709f870e9eec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:11.477531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-650cbdbc-7fe0-40b6-8a1e-709f870e9eec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:11.548844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-650cbdbc-7fe0-40b6-8a1e-709f870e9eec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:11.557366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-650cbdbc-7fe0-40b6-8a1e-709f870e9eec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.095223 Aug 07 01:32:11.557990 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 917/1660] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:11 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:12.581776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b7f26874-45a9-4487-90fa-44fe9c3edde6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:12.582657 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b7f26874-45a9-4487-90fa-44fe9c3edde6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:12.590255 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b7f26874-45a9-4487-90fa-44fe9c3edde6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:12.590812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b7f26874-45a9-4487-90fa-44fe9c3edde6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:12.591292 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b7f26874-45a9-4487-90fa-44fe9c3edde6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:12.592075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b7f26874-45a9-4487-90fa-44fe9c3edde6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:12.592713 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b7f26874-45a9-4487-90fa-44fe9c3edde6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:12.593215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b7f26874-45a9-4487-90fa-44fe9c3edde6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:12.686164 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b7f26874-45a9-4487-90fa-44fe9c3edde6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:12.696973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b7f26874-45a9-4487-90fa-44fe9c3edde6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.120670 Aug 07 01:32:12.697667 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 744/1661] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:12 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:13.728583 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7043bef5-af6b-4409-85d0-66a05d8bfa98 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:13.729459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7043bef5-af6b-4409-85d0-66a05d8bfa98 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:13.736784 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7043bef5-af6b-4409-85d0-66a05d8bfa98 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:13.737273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7043bef5-af6b-4409-85d0-66a05d8bfa98 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:13.737713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7043bef5-af6b-4409-85d0-66a05d8bfa98 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:13.738433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7043bef5-af6b-4409-85d0-66a05d8bfa98 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:13.739009 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7043bef5-af6b-4409-85d0-66a05d8bfa98 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:13.739534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7043bef5-af6b-4409-85d0-66a05d8bfa98 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:13.814417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7043bef5-af6b-4409-85d0-66a05d8bfa98 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:13.825911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7043bef5-af6b-4409-85d0-66a05d8bfa98 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.101952 Aug 07 01:32:13.826580 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 918/1662] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:13 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:14.933470 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-233bf6f0-99d1-4b9e-9d66-4471192dd47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:14.934634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-233bf6f0-99d1-4b9e-9d66-4471192dd47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:14.946559 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-233bf6f0-99d1-4b9e-9d66-4471192dd47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:14.947456 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-233bf6f0-99d1-4b9e-9d66-4471192dd47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:14.948276 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-233bf6f0-99d1-4b9e-9d66-4471192dd47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:14.949547 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-233bf6f0-99d1-4b9e-9d66-4471192dd47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:14.950399 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-233bf6f0-99d1-4b9e-9d66-4471192dd47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:14.951180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-233bf6f0-99d1-4b9e-9d66-4471192dd47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:15.045271 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-233bf6f0-99d1-4b9e-9d66-4471192dd47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:15.054866 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-233bf6f0-99d1-4b9e-9d66-4471192dd47c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.208190 Aug 07 01:32:15.055531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 745/1663] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:14 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:16.078157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-493f64c7-95db-4ade-bcc4-0da7c6f884a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:16.079657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-493f64c7-95db-4ade-bcc4-0da7c6f884a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:16.087581 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-493f64c7-95db-4ade-bcc4-0da7c6f884a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:16.088074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-493f64c7-95db-4ade-bcc4-0da7c6f884a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:16.088548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-493f64c7-95db-4ade-bcc4-0da7c6f884a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:16.089282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-493f64c7-95db-4ade-bcc4-0da7c6f884a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:16.089737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-493f64c7-95db-4ade-bcc4-0da7c6f884a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:16.090163 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-493f64c7-95db-4ade-bcc4-0da7c6f884a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:16.181456 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-493f64c7-95db-4ade-bcc4-0da7c6f884a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:16.191334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-493f64c7-95db-4ade-bcc4-0da7c6f884a8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.118378 Aug 07 01:32:16.192040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 919/1664] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:16 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:17.226854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7db5616c-97dc-4272-8f06-035cdf34a9c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:17.227499 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7db5616c-97dc-4272-8f06-035cdf34a9c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:17.235454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7db5616c-97dc-4272-8f06-035cdf34a9c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:17.236095 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7db5616c-97dc-4272-8f06-035cdf34a9c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:17.236609 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7db5616c-97dc-4272-8f06-035cdf34a9c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:17.237380 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7db5616c-97dc-4272-8f06-035cdf34a9c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:17.237860 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7db5616c-97dc-4272-8f06-035cdf34a9c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:17.238295 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7db5616c-97dc-4272-8f06-035cdf34a9c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:17.322412 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7db5616c-97dc-4272-8f06-035cdf34a9c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:17.332060 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7db5616c-97dc-4272-8f06-035cdf34a9c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.111408 Aug 07 01:32:17.332699 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 746/1665] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:17 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:18.360400 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0fc246f3-d212-4958-812f-787341f8b9b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:18.361660 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0fc246f3-d212-4958-812f-787341f8b9b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:18.375726 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0fc246f3-d212-4958-812f-787341f8b9b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:18.376609 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0fc246f3-d212-4958-812f-787341f8b9b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:18.377696 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0fc246f3-d212-4958-812f-787341f8b9b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:18.379159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0fc246f3-d212-4958-812f-787341f8b9b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:18.380074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0fc246f3-d212-4958-812f-787341f8b9b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:18.381028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0fc246f3-d212-4958-812f-787341f8b9b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:18.634030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0fc246f3-d212-4958-812f-787341f8b9b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:18.644830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0fc246f3-d212-4958-812f-787341f8b9b3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.292261 Aug 07 01:32:18.645494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 920/1666] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:18 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:19.669529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b137c082-b5b4-49f8-86e2-9188b75b6751 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:19.671329 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b137c082-b5b4-49f8-86e2-9188b75b6751 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:19.681156 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b137c082-b5b4-49f8-86e2-9188b75b6751 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:19.681764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b137c082-b5b4-49f8-86e2-9188b75b6751 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:19.682249 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b137c082-b5b4-49f8-86e2-9188b75b6751 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:19.683023 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b137c082-b5b4-49f8-86e2-9188b75b6751 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:19.683726 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b137c082-b5b4-49f8-86e2-9188b75b6751 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:19.684123 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b137c082-b5b4-49f8-86e2-9188b75b6751 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:19.799054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b137c082-b5b4-49f8-86e2-9188b75b6751 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:19.809154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b137c082-b5b4-49f8-86e2-9188b75b6751 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.144773 Aug 07 01:32:19.809759 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 747/1667] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:19 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:20.832359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f4f2378d-57cc-49d2-8552-dc4d68d554ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:20.833054 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f4f2378d-57cc-49d2-8552-dc4d68d554ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:20.839997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4f2378d-57cc-49d2-8552-dc4d68d554ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:20.840526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4f2378d-57cc-49d2-8552-dc4d68d554ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:20.840974 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4f2378d-57cc-49d2-8552-dc4d68d554ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:20.841731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4f2378d-57cc-49d2-8552-dc4d68d554ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:20.842197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4f2378d-57cc-49d2-8552-dc4d68d554ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:20.842633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4f2378d-57cc-49d2-8552-dc4d68d554ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:20.924870 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f4f2378d-57cc-49d2-8552-dc4d68d554ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:20.937986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f4f2378d-57cc-49d2-8552-dc4d68d554ce tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.110270 Aug 07 01:32:20.938880 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 921/1668] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:20 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:21.967146 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-09fdbea6-36d2-4a73-a7c6-08a9a812f196 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:21.968836 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-09fdbea6-36d2-4a73-a7c6-08a9a812f196 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:21.980667 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09fdbea6-36d2-4a73-a7c6-08a9a812f196 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:21.981638 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09fdbea6-36d2-4a73-a7c6-08a9a812f196 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:21.982456 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09fdbea6-36d2-4a73-a7c6-08a9a812f196 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:21.984441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09fdbea6-36d2-4a73-a7c6-08a9a812f196 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:21.985363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09fdbea6-36d2-4a73-a7c6-08a9a812f196 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:21.986182 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-09fdbea6-36d2-4a73-a7c6-08a9a812f196 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:22.073644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-09fdbea6-36d2-4a73-a7c6-08a9a812f196 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:22.082593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-09fdbea6-36d2-4a73-a7c6-08a9a812f196 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.123643 Aug 07 01:32:22.083196 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 748/1669] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:21 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:23.124455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1bbdaa00-5301-47c6-a547-d12d717bddc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:23.125650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1bbdaa00-5301-47c6-a547-d12d717bddc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:23.138821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1bbdaa00-5301-47c6-a547-d12d717bddc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:23.139715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1bbdaa00-5301-47c6-a547-d12d717bddc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:23.141189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1bbdaa00-5301-47c6-a547-d12d717bddc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:23.142525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1bbdaa00-5301-47c6-a547-d12d717bddc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:23.143374 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1bbdaa00-5301-47c6-a547-d12d717bddc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:23.144170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1bbdaa00-5301-47c6-a547-d12d717bddc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:23.233040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1bbdaa00-5301-47c6-a547-d12d717bddc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:23.243015 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1bbdaa00-5301-47c6-a547-d12d717bddc7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.126314 Aug 07 01:32:23.243776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 922/1670] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:23 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:24.275114 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d4059733-10e9-4548-b138-aa19c5d1a226 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:24.275823 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d4059733-10e9-4548-b138-aa19c5d1a226 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:24.284624 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d4059733-10e9-4548-b138-aa19c5d1a226 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:24.285296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d4059733-10e9-4548-b138-aa19c5d1a226 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:24.285845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d4059733-10e9-4548-b138-aa19c5d1a226 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:24.286785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d4059733-10e9-4548-b138-aa19c5d1a226 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:24.287513 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d4059733-10e9-4548-b138-aa19c5d1a226 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:24.288028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d4059733-10e9-4548-b138-aa19c5d1a226 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:24.362339 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d4059733-10e9-4548-b138-aa19c5d1a226 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:24.372076 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d4059733-10e9-4548-b138-aa19c5d1a226 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.101213 Aug 07 01:32:24.372828 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 749/1671] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:24 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:25.394367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1c01e596-5460-44a3-a821-d71ce71e5fbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:25.394999 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1c01e596-5460-44a3-a821-d71ce71e5fbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:25.402044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c01e596-5460-44a3-a821-d71ce71e5fbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:25.402541 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c01e596-5460-44a3-a821-d71ce71e5fbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:25.403010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c01e596-5460-44a3-a821-d71ce71e5fbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:25.403836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c01e596-5460-44a3-a821-d71ce71e5fbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:25.404341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c01e596-5460-44a3-a821-d71ce71e5fbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:25.404782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1c01e596-5460-44a3-a821-d71ce71e5fbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:25.408492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=84 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:32:25.485346 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1c01e596-5460-44a3-a821-d71ce71e5fbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:25.494168 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1c01e596-5460-44a3-a821-d71ce71e5fbd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.104193 Aug 07 01:32:25.494903 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 923/1672] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:25 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:26.529878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c7d87dfc-543c-4a59-b0b7-92662cf084c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:26.531209 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c7d87dfc-543c-4a59-b0b7-92662cf084c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:26.537975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7d87dfc-543c-4a59-b0b7-92662cf084c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:26.538377 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7d87dfc-543c-4a59-b0b7-92662cf084c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:26.538822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7d87dfc-543c-4a59-b0b7-92662cf084c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:26.539621 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7d87dfc-543c-4a59-b0b7-92662cf084c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:26.540076 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7d87dfc-543c-4a59-b0b7-92662cf084c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:26.540570 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7d87dfc-543c-4a59-b0b7-92662cf084c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:26.620655 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c7d87dfc-543c-4a59-b0b7-92662cf084c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:26.635032 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c7d87dfc-543c-4a59-b0b7-92662cf084c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.108902 Aug 07 01:32:26.635856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 750/1673] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:26 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:26.958928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:32:26.959233 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:32:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:32:27.658844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f238ebd8-ad72-451a-8428-06300d3737ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:27.659782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f238ebd8-ad72-451a-8428-06300d3737ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:27.662442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=28 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:32:27.669366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f238ebd8-ad72-451a-8428-06300d3737ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:27.669972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f238ebd8-ad72-451a-8428-06300d3737ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:27.670424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f238ebd8-ad72-451a-8428-06300d3737ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:27.671803 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f238ebd8-ad72-451a-8428-06300d3737ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:27.672324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f238ebd8-ad72-451a-8428-06300d3737ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:27.672862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f238ebd8-ad72-451a-8428-06300d3737ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:27.780776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f238ebd8-ad72-451a-8428-06300d3737ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:27.796740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f238ebd8-ad72-451a-8428-06300d3737ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.141560 Aug 07 01:32:27.797812 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 924/1674] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:27 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:28.020900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:32:28.021280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:32:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:32:28.830993 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e153f853-7b0c-464f-ba33-25191392d83e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:28.831689 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e153f853-7b0c-464f-ba33-25191392d83e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:28.834400 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=28 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:32:28.840301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e153f853-7b0c-464f-ba33-25191392d83e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:28.840863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e153f853-7b0c-464f-ba33-25191392d83e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:28.841351 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e153f853-7b0c-464f-ba33-25191392d83e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:28.842222 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e153f853-7b0c-464f-ba33-25191392d83e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:28.843323 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e153f853-7b0c-464f-ba33-25191392d83e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:28.843841 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e153f853-7b0c-464f-ba33-25191392d83e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:28.847898 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=84 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:32:28.934688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e153f853-7b0c-464f-ba33-25191392d83e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:28.945251 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e153f853-7b0c-464f-ba33-25191392d83e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.118068 Aug 07 01:32:28.946303 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 751/1675] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:28 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:29.987848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6686106c-b8c8-4e16-b4e4-918a0be8de71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:29.989063 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6686106c-b8c8-4e16-b4e4-918a0be8de71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:30.000126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6686106c-b8c8-4e16-b4e4-918a0be8de71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:30.000910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6686106c-b8c8-4e16-b4e4-918a0be8de71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:30.001562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6686106c-b8c8-4e16-b4e4-918a0be8de71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:30.002497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6686106c-b8c8-4e16-b4e4-918a0be8de71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:30.003066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6686106c-b8c8-4e16-b4e4-918a0be8de71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:30.003587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6686106c-b8c8-4e16-b4e4-918a0be8de71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:30.107953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6686106c-b8c8-4e16-b4e4-918a0be8de71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:30.117899 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6686106c-b8c8-4e16-b4e4-918a0be8de71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.136080 Aug 07 01:32:30.118535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 925/1676] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:29 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:31.144391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6fb822e8-ce5f-4a22-ad24-583a6490f1e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:31.145522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6fb822e8-ce5f-4a22-ad24-583a6490f1e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:31.153835 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6fb822e8-ce5f-4a22-ad24-583a6490f1e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:31.154385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6fb822e8-ce5f-4a22-ad24-583a6490f1e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:31.155052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6fb822e8-ce5f-4a22-ad24-583a6490f1e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:31.155947 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6fb822e8-ce5f-4a22-ad24-583a6490f1e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:31.156575 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6fb822e8-ce5f-4a22-ad24-583a6490f1e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:31.157227 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6fb822e8-ce5f-4a22-ad24-583a6490f1e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:31.241019 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6fb822e8-ce5f-4a22-ad24-583a6490f1e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:31.257564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6fb822e8-ce5f-4a22-ad24-583a6490f1e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.118989 Aug 07 01:32:31.258667 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 752/1677] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:31 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:32.291920 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ced0d75f-580f-4bc4-a891-bd89a7a6e585 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:32.293075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ced0d75f-580f-4bc4-a891-bd89a7a6e585 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:32.302162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ced0d75f-580f-4bc4-a891-bd89a7a6e585 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:32.302866 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ced0d75f-580f-4bc4-a891-bd89a7a6e585 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:32.303432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ced0d75f-580f-4bc4-a891-bd89a7a6e585 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:32.304403 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ced0d75f-580f-4bc4-a891-bd89a7a6e585 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:32.305011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ced0d75f-580f-4bc4-a891-bd89a7a6e585 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:32.305596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ced0d75f-580f-4bc4-a891-bd89a7a6e585 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:32.394423 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ced0d75f-580f-4bc4-a891-bd89a7a6e585 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:32.402945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ced0d75f-580f-4bc4-a891-bd89a7a6e585 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.115459 Aug 07 01:32:32.403528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 926/1678] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:32 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:33.426481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d5a303fd-4040-4be4-b302-68cf521e97dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:33.427258 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d5a303fd-4040-4be4-b302-68cf521e97dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:33.434830 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d5a303fd-4040-4be4-b302-68cf521e97dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:33.435388 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d5a303fd-4040-4be4-b302-68cf521e97dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:33.435871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d5a303fd-4040-4be4-b302-68cf521e97dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:33.436713 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d5a303fd-4040-4be4-b302-68cf521e97dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:33.437234 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d5a303fd-4040-4be4-b302-68cf521e97dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:33.437717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d5a303fd-4040-4be4-b302-68cf521e97dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:33.516777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d5a303fd-4040-4be4-b302-68cf521e97dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:33.530352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d5a303fd-4040-4be4-b302-68cf521e97dd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.107753 Aug 07 01:32:33.531167 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 753/1679] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:33 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:34.553191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b03ae0ad-6a2c-4400-9720-30ede3cb41f3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:34.554025 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b03ae0ad-6a2c-4400-9720-30ede3cb41f3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:34.562224 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b03ae0ad-6a2c-4400-9720-30ede3cb41f3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:34.562865 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b03ae0ad-6a2c-4400-9720-30ede3cb41f3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:34.563509 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b03ae0ad-6a2c-4400-9720-30ede3cb41f3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:34.564439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b03ae0ad-6a2c-4400-9720-30ede3cb41f3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:34.565014 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b03ae0ad-6a2c-4400-9720-30ede3cb41f3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:34.565685 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b03ae0ad-6a2c-4400-9720-30ede3cb41f3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:34.651007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b03ae0ad-6a2c-4400-9720-30ede3cb41f3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:34.669678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b03ae0ad-6a2c-4400-9720-30ede3cb41f3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.120513 Aug 07 01:32:34.675424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 927/1680] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:34 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:35.703566 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-56eda404-5d72-481d-89db-c8993b2a267c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:35.704224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-56eda404-5d72-481d-89db-c8993b2a267c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:35.712116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-56eda404-5d72-481d-89db-c8993b2a267c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:35.712842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-56eda404-5d72-481d-89db-c8993b2a267c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:35.713420 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-56eda404-5d72-481d-89db-c8993b2a267c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:35.714328 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-56eda404-5d72-481d-89db-c8993b2a267c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:35.714849 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-56eda404-5d72-481d-89db-c8993b2a267c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:35.715329 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-56eda404-5d72-481d-89db-c8993b2a267c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:35.790249 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-56eda404-5d72-481d-89db-c8993b2a267c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:35.799417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-56eda404-5d72-481d-89db-c8993b2a267c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.103373 Aug 07 01:32:35.800020 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 754/1681] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:35 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:36.833204 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-de9e080c-ca28-421d-935b-252dc03d0d17 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:36.834068 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-de9e080c-ca28-421d-935b-252dc03d0d17 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:36.847736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-de9e080c-ca28-421d-935b-252dc03d0d17 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:36.848615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-de9e080c-ca28-421d-935b-252dc03d0d17 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:36.849096 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-de9e080c-ca28-421d-935b-252dc03d0d17 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:36.849936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-de9e080c-ca28-421d-935b-252dc03d0d17 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:36.850402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-de9e080c-ca28-421d-935b-252dc03d0d17 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:36.850835 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-de9e080c-ca28-421d-935b-252dc03d0d17 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:36.921492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-de9e080c-ca28-421d-935b-252dc03d0d17 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:36.930593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-de9e080c-ca28-421d-935b-252dc03d0d17 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.101541 Aug 07 01:32:36.932264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 928/1682] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:36 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:37.954099 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4adf6baf-0bd2-46a7-a900-9b2a7bbbb3be tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:37.955222 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4adf6baf-0bd2-46a7-a900-9b2a7bbbb3be tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:37.962058 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4adf6baf-0bd2-46a7-a900-9b2a7bbbb3be tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:37.962703 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4adf6baf-0bd2-46a7-a900-9b2a7bbbb3be tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:37.963238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4adf6baf-0bd2-46a7-a900-9b2a7bbbb3be tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:37.964163 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4adf6baf-0bd2-46a7-a900-9b2a7bbbb3be tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:37.964755 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4adf6baf-0bd2-46a7-a900-9b2a7bbbb3be tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:37.965278 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4adf6baf-0bd2-46a7-a900-9b2a7bbbb3be tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:38.043666 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4adf6baf-0bd2-46a7-a900-9b2a7bbbb3be tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:38.054265 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4adf6baf-0bd2-46a7-a900-9b2a7bbbb3be tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.104733 Aug 07 01:32:38.054855 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 755/1683] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:37 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:39.080759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-99db8d90-5e29-484e-a216-0a22265293fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:39.081891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-99db8d90-5e29-484e-a216-0a22265293fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:39.094832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-99db8d90-5e29-484e-a216-0a22265293fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:39.095706 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-99db8d90-5e29-484e-a216-0a22265293fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:39.096537 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-99db8d90-5e29-484e-a216-0a22265293fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:39.097869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-99db8d90-5e29-484e-a216-0a22265293fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:39.098722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-99db8d90-5e29-484e-a216-0a22265293fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:39.099524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-99db8d90-5e29-484e-a216-0a22265293fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:39.206283 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-99db8d90-5e29-484e-a216-0a22265293fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:39.219110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-99db8d90-5e29-484e-a216-0a22265293fe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.143912 Aug 07 01:32:39.219110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 929/1684] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:39 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:40.250760 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-92b9a7c0-00f7-45fe-a9ff-58a1346c0bf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:40.251546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-92b9a7c0-00f7-45fe-a9ff-58a1346c0bf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:40.260596 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-92b9a7c0-00f7-45fe-a9ff-58a1346c0bf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:40.261481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-92b9a7c0-00f7-45fe-a9ff-58a1346c0bf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:40.261968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-92b9a7c0-00f7-45fe-a9ff-58a1346c0bf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:40.262834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-92b9a7c0-00f7-45fe-a9ff-58a1346c0bf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:40.263293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-92b9a7c0-00f7-45fe-a9ff-58a1346c0bf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:40.263881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-92b9a7c0-00f7-45fe-a9ff-58a1346c0bf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:40.343494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-92b9a7c0-00f7-45fe-a9ff-58a1346c0bf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:40.352409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-92b9a7c0-00f7-45fe-a9ff-58a1346c0bf0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.106850 Aug 07 01:32:40.353012 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 756/1685] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:40 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:41.375185 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e2f09176-362e-4b8c-a4e6-d3bdb8d9a2ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:41.375838 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e2f09176-362e-4b8c-a4e6-d3bdb8d9a2ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:41.386114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e2f09176-362e-4b8c-a4e6-d3bdb8d9a2ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:41.386936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e2f09176-362e-4b8c-a4e6-d3bdb8d9a2ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:41.387654 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e2f09176-362e-4b8c-a4e6-d3bdb8d9a2ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:41.388738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e2f09176-362e-4b8c-a4e6-d3bdb8d9a2ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:41.389274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e2f09176-362e-4b8c-a4e6-d3bdb8d9a2ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:41.389802 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e2f09176-362e-4b8c-a4e6-d3bdb8d9a2ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:41.474823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e2f09176-362e-4b8c-a4e6-d3bdb8d9a2ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:41.484028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e2f09176-362e-4b8c-a4e6-d3bdb8d9a2ff tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.113774 Aug 07 01:32:41.484692 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 930/1686] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:41 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:42.506661 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d38c35bd-187c-4b8c-94b8-c3887a34ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:42.507334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d38c35bd-187c-4b8c-94b8-c3887a34ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:42.515141 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d38c35bd-187c-4b8c-94b8-c3887a34ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:42.515638 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d38c35bd-187c-4b8c-94b8-c3887a34ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:42.516090 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d38c35bd-187c-4b8c-94b8-c3887a34ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:42.516856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d38c35bd-187c-4b8c-94b8-c3887a34ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:42.517322 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d38c35bd-187c-4b8c-94b8-c3887a34ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:42.517789 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d38c35bd-187c-4b8c-94b8-c3887a34ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:42.597965 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d38c35bd-187c-4b8c-94b8-c3887a34ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:42.606591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d38c35bd-187c-4b8c-94b8-c3887a34ea08 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.104090 Aug 07 01:32:42.607268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 757/1687] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:42 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:43.630257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e4de73c8-9199-4dc8-a73a-e4e03f1003ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:43.630883 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e4de73c8-9199-4dc8-a73a-e4e03f1003ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:43.638278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4de73c8-9199-4dc8-a73a-e4e03f1003ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:43.639057 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4de73c8-9199-4dc8-a73a-e4e03f1003ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:43.639410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4de73c8-9199-4dc8-a73a-e4e03f1003ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:43.640445 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4de73c8-9199-4dc8-a73a-e4e03f1003ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:43.641001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4de73c8-9199-4dc8-a73a-e4e03f1003ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:43.641530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4de73c8-9199-4dc8-a73a-e4e03f1003ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:43.716973 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e4de73c8-9199-4dc8-a73a-e4e03f1003ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:43.732472 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e4de73c8-9199-4dc8-a73a-e4e03f1003ca tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.105989 Aug 07 01:32:43.734132 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 931/1688] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:43 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:44.767753 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0bbe42cd-5f34-4958-8f90-0799182cdfbb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:44.768752 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0bbe42cd-5f34-4958-8f90-0799182cdfbb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:44.777421 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0bbe42cd-5f34-4958-8f90-0799182cdfbb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:44.778281 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0bbe42cd-5f34-4958-8f90-0799182cdfbb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:44.779173 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0bbe42cd-5f34-4958-8f90-0799182cdfbb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:44.780563 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0bbe42cd-5f34-4958-8f90-0799182cdfbb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:44.780980 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0bbe42cd-5f34-4958-8f90-0799182cdfbb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:44.781655 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0bbe42cd-5f34-4958-8f90-0799182cdfbb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:44.852677 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0bbe42cd-5f34-4958-8f90-0799182cdfbb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:44.862771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0bbe42cd-5f34-4958-8f90-0799182cdfbb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.099270 Aug 07 01:32:44.863442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 758/1689] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:44 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:45.885339 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-04e6e596-e022-4720-b872-bb8a1712f240 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:45.886126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-04e6e596-e022-4720-b872-bb8a1712f240 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:45.894201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04e6e596-e022-4720-b872-bb8a1712f240 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:45.894703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04e6e596-e022-4720-b872-bb8a1712f240 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:45.895153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04e6e596-e022-4720-b872-bb8a1712f240 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:45.895888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04e6e596-e022-4720-b872-bb8a1712f240 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:45.896384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04e6e596-e022-4720-b872-bb8a1712f240 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:45.896816 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04e6e596-e022-4720-b872-bb8a1712f240 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:45.977143 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-04e6e596-e022-4720-b872-bb8a1712f240 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:45.986173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-04e6e596-e022-4720-b872-bb8a1712f240 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.105237 Aug 07 01:32:45.986781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 932/1690] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:45 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:47.009775 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6f71a291-579e-457e-9e96-eb602b2c104f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:47.010732 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6f71a291-579e-457e-9e96-eb602b2c104f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:47.018040 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6f71a291-579e-457e-9e96-eb602b2c104f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:47.018605 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6f71a291-579e-457e-9e96-eb602b2c104f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:47.019101 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6f71a291-579e-457e-9e96-eb602b2c104f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:47.019887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6f71a291-579e-457e-9e96-eb602b2c104f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:47.020485 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6f71a291-579e-457e-9e96-eb602b2c104f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:47.020975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6f71a291-579e-457e-9e96-eb602b2c104f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:47.121102 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6f71a291-579e-457e-9e96-eb602b2c104f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:47.138491 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6f71a291-579e-457e-9e96-eb602b2c104f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.132944 Aug 07 01:32:47.140020 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 759/1691] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:47 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:48.162528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-deafb440-5c39-48d7-8392-92fd4ceef5fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:48.163533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-deafb440-5c39-48d7-8392-92fd4ceef5fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:48.170997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-deafb440-5c39-48d7-8392-92fd4ceef5fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:48.171500 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-deafb440-5c39-48d7-8392-92fd4ceef5fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:48.171944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-deafb440-5c39-48d7-8392-92fd4ceef5fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:48.172715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-deafb440-5c39-48d7-8392-92fd4ceef5fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:48.173174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-deafb440-5c39-48d7-8392-92fd4ceef5fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:48.173608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-deafb440-5c39-48d7-8392-92fd4ceef5fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:48.245985 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-deafb440-5c39-48d7-8392-92fd4ceef5fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:48.255834 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-deafb440-5c39-48d7-8392-92fd4ceef5fc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097665 Aug 07 01:32:48.256486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 933/1692] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:48 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:49.279519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-862923be-8d9b-4a09-9ac7-f0a001cca01b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:49.281103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-862923be-8d9b-4a09-9ac7-f0a001cca01b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:49.289175 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-862923be-8d9b-4a09-9ac7-f0a001cca01b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:49.289716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-862923be-8d9b-4a09-9ac7-f0a001cca01b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:49.290239 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-862923be-8d9b-4a09-9ac7-f0a001cca01b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:49.291018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-862923be-8d9b-4a09-9ac7-f0a001cca01b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:49.291529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-862923be-8d9b-4a09-9ac7-f0a001cca01b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:49.292014 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-862923be-8d9b-4a09-9ac7-f0a001cca01b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:49.378938 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-862923be-8d9b-4a09-9ac7-f0a001cca01b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:49.388532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-862923be-8d9b-4a09-9ac7-f0a001cca01b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.114428 Aug 07 01:32:49.389855 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 760/1693] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:49 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:50.412261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6b47f8da-14b6-44b7-b52a-1c8ebc8b7bae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:50.413303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6b47f8da-14b6-44b7-b52a-1c8ebc8b7bae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:50.420204 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6b47f8da-14b6-44b7-b52a-1c8ebc8b7bae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:50.421338 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6b47f8da-14b6-44b7-b52a-1c8ebc8b7bae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:50.421834 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6b47f8da-14b6-44b7-b52a-1c8ebc8b7bae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:50.422681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6b47f8da-14b6-44b7-b52a-1c8ebc8b7bae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:50.423286 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6b47f8da-14b6-44b7-b52a-1c8ebc8b7bae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:50.423834 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6b47f8da-14b6-44b7-b52a-1c8ebc8b7bae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:50.497177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6b47f8da-14b6-44b7-b52a-1c8ebc8b7bae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:50.506517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6b47f8da-14b6-44b7-b52a-1c8ebc8b7bae tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098384 Aug 07 01:32:50.507174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 934/1694] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:50 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:51.527418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0618c9d6-277e-4856-aca8-c1d4ee3551d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:51.528144 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0618c9d6-277e-4856-aca8-c1d4ee3551d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:51.535861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0618c9d6-277e-4856-aca8-c1d4ee3551d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:51.536310 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0618c9d6-277e-4856-aca8-c1d4ee3551d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:51.536759 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0618c9d6-277e-4856-aca8-c1d4ee3551d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:51.537497 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0618c9d6-277e-4856-aca8-c1d4ee3551d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:51.537954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0618c9d6-277e-4856-aca8-c1d4ee3551d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:51.538385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0618c9d6-277e-4856-aca8-c1d4ee3551d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:51.608100 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0618c9d6-277e-4856-aca8-c1d4ee3551d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:51.618944 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0618c9d6-277e-4856-aca8-c1d4ee3551d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.095379 Aug 07 01:32:51.619550 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 761/1695] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:51 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:32:52.640979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a540079c-3283-48cd-954d-ac023b0f882a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:52.641681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a540079c-3283-48cd-954d-ac023b0f882a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:52.652715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a540079c-3283-48cd-954d-ac023b0f882a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:52.653251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a540079c-3283-48cd-954d-ac023b0f882a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:52.653693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a540079c-3283-48cd-954d-ac023b0f882a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:52.654508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a540079c-3283-48cd-954d-ac023b0f882a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:52.655061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a540079c-3283-48cd-954d-ac023b0f882a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:52.656189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a540079c-3283-48cd-954d-ac023b0f882a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:52.724947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a540079c-3283-48cd-954d-ac023b0f882a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:52.733335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a540079c-3283-48cd-954d-ac023b0f882a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097665 Aug 07 01:32:52.733921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 935/1696] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:52 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:53.756105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-37e66845-1130-424a-95e8-5b89d84281c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:53.756855 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-37e66845-1130-424a-95e8-5b89d84281c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:53.764761 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37e66845-1130-424a-95e8-5b89d84281c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:53.765382 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37e66845-1130-424a-95e8-5b89d84281c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:53.765846 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37e66845-1130-424a-95e8-5b89d84281c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:53.766577 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37e66845-1130-424a-95e8-5b89d84281c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:53.767065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37e66845-1130-424a-95e8-5b89d84281c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:53.767534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-37e66845-1130-424a-95e8-5b89d84281c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:53.857195 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-37e66845-1130-424a-95e8-5b89d84281c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:53.872149 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-37e66845-1130-424a-95e8-5b89d84281c9 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.120149 Aug 07 01:32:53.873258 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 762/1697] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:53 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:54.907055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-78f5c17d-bb1a-41ca-9259-128e5bf4ff92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:54.907959 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-78f5c17d-bb1a-41ca-9259-128e5bf4ff92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:54.915227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78f5c17d-bb1a-41ca-9259-128e5bf4ff92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:54.915823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78f5c17d-bb1a-41ca-9259-128e5bf4ff92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:54.916305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78f5c17d-bb1a-41ca-9259-128e5bf4ff92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:54.917039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78f5c17d-bb1a-41ca-9259-128e5bf4ff92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:54.917502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78f5c17d-bb1a-41ca-9259-128e5bf4ff92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:54.918075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78f5c17d-bb1a-41ca-9259-128e5bf4ff92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:54.996752 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-78f5c17d-bb1a-41ca-9259-128e5bf4ff92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:55.006378 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-78f5c17d-bb1a-41ca-9259-128e5bf4ff92 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.103895 Aug 07 01:32:55.007138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 936/1698] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:54 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:56.028782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6c0455a8-c44c-4293-b761-93f65aa3960a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:56.029543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6c0455a8-c44c-4293-b761-93f65aa3960a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:56.037968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c0455a8-c44c-4293-b761-93f65aa3960a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:56.038523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c0455a8-c44c-4293-b761-93f65aa3960a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:56.039004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c0455a8-c44c-4293-b761-93f65aa3960a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:56.039863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c0455a8-c44c-4293-b761-93f65aa3960a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:56.040410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c0455a8-c44c-4293-b761-93f65aa3960a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:56.041004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6c0455a8-c44c-4293-b761-93f65aa3960a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:56.117795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6c0455a8-c44c-4293-b761-93f65aa3960a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:56.131939 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6c0455a8-c44c-4293-b761-93f65aa3960a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.107462 Aug 07 01:32:56.132878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 763/1699] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:56 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:57.165423 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f8ed7e11-7dfd-415d-a8f6-d6988f5aa9db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:57.166087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f8ed7e11-7dfd-415d-a8f6-d6988f5aa9db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:57.174763 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8ed7e11-7dfd-415d-a8f6-d6988f5aa9db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:57.175442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8ed7e11-7dfd-415d-a8f6-d6988f5aa9db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:57.176111 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8ed7e11-7dfd-415d-a8f6-d6988f5aa9db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:57.176913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8ed7e11-7dfd-415d-a8f6-d6988f5aa9db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:57.177384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8ed7e11-7dfd-415d-a8f6-d6988f5aa9db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:57.177824 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8ed7e11-7dfd-415d-a8f6-d6988f5aa9db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:57.181433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=84 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:32:57.263705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f8ed7e11-7dfd-415d-a8f6-d6988f5aa9db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:57.272751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f8ed7e11-7dfd-415d-a8f6-d6988f5aa9db tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.111300 Aug 07 01:32:57.273349 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 937/1700] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:57 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:58.295933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a35ec7e9-a167-4408-858c-0bea858bb09e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:58.296866 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a35ec7e9-a167-4408-858c-0bea858bb09e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:58.304092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a35ec7e9-a167-4408-858c-0bea858bb09e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:58.304616 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a35ec7e9-a167-4408-858c-0bea858bb09e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:58.305058 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a35ec7e9-a167-4408-858c-0bea858bb09e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:58.305796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a35ec7e9-a167-4408-858c-0bea858bb09e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:58.306252 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a35ec7e9-a167-4408-858c-0bea858bb09e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:58.306683 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a35ec7e9-a167-4408-858c-0bea858bb09e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:58.379075 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a35ec7e9-a167-4408-858c-0bea858bb09e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:58.389132 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a35ec7e9-a167-4408-858c-0bea858bb09e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097785 Aug 07 01:32:58.389834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 764/1701] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:58 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:32:59.412273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-685b03a2-f370-4ef6-9fe7-d136e8b3a6dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:32:59.412926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-685b03a2-f370-4ef6-9fe7-d136e8b3a6dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:32:59.415693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=28 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:32:59.422986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-685b03a2-f370-4ef6-9fe7-d136e8b3a6dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:59.423759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-685b03a2-f370-4ef6-9fe7-d136e8b3a6dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:59.424222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-685b03a2-f370-4ef6-9fe7-d136e8b3a6dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:59.425170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-685b03a2-f370-4ef6-9fe7-d136e8b3a6dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:32:59.425706 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-685b03a2-f370-4ef6-9fe7-d136e8b3a6dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:32:59.426367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-685b03a2-f370-4ef6-9fe7-d136e8b3a6dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:32:59.499335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-685b03a2-f370-4ef6-9fe7-d136e8b3a6dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:32:59.507962 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-685b03a2-f370-4ef6-9fe7-d136e8b3a6dc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.099657 Aug 07 01:32:59.508590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 938/1702] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:32:59 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:00.541565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f265a0dc-f42d-454e-a0a4-a94ab74eac7b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:00.542221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f265a0dc-f42d-454e-a0a4-a94ab74eac7b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:00.547528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=28 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:33:00.557117 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f265a0dc-f42d-454e-a0a4-a94ab74eac7b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:00.557992 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f265a0dc-f42d-454e-a0a4-a94ab74eac7b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:00.558727 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f265a0dc-f42d-454e-a0a4-a94ab74eac7b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:00.560011 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f265a0dc-f42d-454e-a0a4-a94ab74eac7b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:00.560870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f265a0dc-f42d-454e-a0a4-a94ab74eac7b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:00.562354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f265a0dc-f42d-454e-a0a4-a94ab74eac7b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:00.570557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=84 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:33:00.656477 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f265a0dc-f42d-454e-a0a4-a94ab74eac7b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:00.665409 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f265a0dc-f42d-454e-a0a4-a94ab74eac7b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.127916 Aug 07 01:33:00.666007 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 765/1703] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:00 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:01.688215 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e39ea13e-081f-4eee-b492-271feec4574b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:01.688835 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e39ea13e-081f-4eee-b492-271feec4574b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:01.695646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e39ea13e-081f-4eee-b492-271feec4574b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:01.696135 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e39ea13e-081f-4eee-b492-271feec4574b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:01.696603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e39ea13e-081f-4eee-b492-271feec4574b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:01.697492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e39ea13e-081f-4eee-b492-271feec4574b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:01.697777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e39ea13e-081f-4eee-b492-271feec4574b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:01.698203 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e39ea13e-081f-4eee-b492-271feec4574b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:01.806314 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e39ea13e-081f-4eee-b492-271feec4574b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:01.815680 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e39ea13e-081f-4eee-b492-271feec4574b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.131948 Aug 07 01:33:01.816333 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 939/1704] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:01 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:02.850035 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fd230870-7fd0-4ac5-b4d6-d14efa705fef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:02.851066 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fd230870-7fd0-4ac5-b4d6-d14efa705fef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:02.858859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd230870-7fd0-4ac5-b4d6-d14efa705fef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:02.859351 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd230870-7fd0-4ac5-b4d6-d14efa705fef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:02.860065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd230870-7fd0-4ac5-b4d6-d14efa705fef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:02.860856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd230870-7fd0-4ac5-b4d6-d14efa705fef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:02.861327 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd230870-7fd0-4ac5-b4d6-d14efa705fef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:02.861758 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fd230870-7fd0-4ac5-b4d6-d14efa705fef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:02.932422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fd230870-7fd0-4ac5-b4d6-d14efa705fef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:02.942084 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fd230870-7fd0-4ac5-b4d6-d14efa705fef tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.096232 Aug 07 01:33:02.942821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 766/1705] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:02 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:03.965626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-add2d21b-15a3-40da-ab58-9b35fd014d72 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:03.968049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-add2d21b-15a3-40da-ab58-9b35fd014d72 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:03.975558 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-add2d21b-15a3-40da-ab58-9b35fd014d72 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:03.976117 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-add2d21b-15a3-40da-ab58-9b35fd014d72 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:03.976698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-add2d21b-15a3-40da-ab58-9b35fd014d72 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:03.977429 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-add2d21b-15a3-40da-ab58-9b35fd014d72 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:03.977986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-add2d21b-15a3-40da-ab58-9b35fd014d72 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:03.978333 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-add2d21b-15a3-40da-ab58-9b35fd014d72 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:04.069519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-add2d21b-15a3-40da-ab58-9b35fd014d72 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:04.079421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-add2d21b-15a3-40da-ab58-9b35fd014d72 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.119394 Aug 07 01:33:04.080500 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 940/1706] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:03 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:05.115354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-252981d9-0073-4ff1-9180-50fa8b59d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:05.116193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-252981d9-0073-4ff1-9180-50fa8b59d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:05.123179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-252981d9-0073-4ff1-9180-50fa8b59d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:05.124310 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-252981d9-0073-4ff1-9180-50fa8b59d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:05.124763 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-252981d9-0073-4ff1-9180-50fa8b59d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:05.125511 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-252981d9-0073-4ff1-9180-50fa8b59d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:05.125968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-252981d9-0073-4ff1-9180-50fa8b59d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:05.126405 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-252981d9-0073-4ff1-9180-50fa8b59d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:05.199993 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-252981d9-0073-4ff1-9180-50fa8b59d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:05.209137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-252981d9-0073-4ff1-9180-50fa8b59d5c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097710 Aug 07 01:33:05.209737 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 767/1707] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:05 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:06.235611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9d15c400-a458-4d48-8bf0-5f240779e909 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:06.236788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9d15c400-a458-4d48-8bf0-5f240779e909 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:06.248793 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d15c400-a458-4d48-8bf0-5f240779e909 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:06.249561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d15c400-a458-4d48-8bf0-5f240779e909 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:06.250342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d15c400-a458-4d48-8bf0-5f240779e909 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:06.251644 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d15c400-a458-4d48-8bf0-5f240779e909 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:06.252606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d15c400-a458-4d48-8bf0-5f240779e909 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:06.253399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d15c400-a458-4d48-8bf0-5f240779e909 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:06.334140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9d15c400-a458-4d48-8bf0-5f240779e909 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:06.349116 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9d15c400-a458-4d48-8bf0-5f240779e909 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.121373 Aug 07 01:33:06.350264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 941/1708] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:06 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:07.383242 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9ecf42a2-07b0-49bc-8130-5057ed6157d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:07.383922 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9ecf42a2-07b0-49bc-8130-5057ed6157d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:07.391457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ecf42a2-07b0-49bc-8130-5057ed6157d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:07.392000 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ecf42a2-07b0-49bc-8130-5057ed6157d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:07.392634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ecf42a2-07b0-49bc-8130-5057ed6157d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:07.393494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ecf42a2-07b0-49bc-8130-5057ed6157d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:07.394046 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ecf42a2-07b0-49bc-8130-5057ed6157d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:07.394566 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ecf42a2-07b0-49bc-8130-5057ed6157d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:07.468092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9ecf42a2-07b0-49bc-8130-5057ed6157d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:07.477747 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9ecf42a2-07b0-49bc-8130-5057ed6157d6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098412 Aug 07 01:33:07.478348 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 768/1709] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:07 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:08.511873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8d1aa25b-fc60-4a35-bd97-98b21d6a844e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:08.512652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8d1aa25b-fc60-4a35-bd97-98b21d6a844e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:08.520702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8d1aa25b-fc60-4a35-bd97-98b21d6a844e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:08.521200 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8d1aa25b-fc60-4a35-bd97-98b21d6a844e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:08.521648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8d1aa25b-fc60-4a35-bd97-98b21d6a844e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:08.522394 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8d1aa25b-fc60-4a35-bd97-98b21d6a844e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:08.522857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8d1aa25b-fc60-4a35-bd97-98b21d6a844e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:08.523290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8d1aa25b-fc60-4a35-bd97-98b21d6a844e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:08.609941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8d1aa25b-fc60-4a35-bd97-98b21d6a844e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:08.618726 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8d1aa25b-fc60-4a35-bd97-98b21d6a844e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.111258 Aug 07 01:33:08.619441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 942/1710] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:08 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:09.642208 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-eb6445d8-7695-411d-acd3-324dd9b38150 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:09.642992 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-eb6445d8-7695-411d-acd3-324dd9b38150 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:09.650966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb6445d8-7695-411d-acd3-324dd9b38150 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:09.651579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb6445d8-7695-411d-acd3-324dd9b38150 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:09.652082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb6445d8-7695-411d-acd3-324dd9b38150 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:09.652985 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb6445d8-7695-411d-acd3-324dd9b38150 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:09.653576 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb6445d8-7695-411d-acd3-324dd9b38150 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:09.654061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-eb6445d8-7695-411d-acd3-324dd9b38150 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:09.725379 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-eb6445d8-7695-411d-acd3-324dd9b38150 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:09.734844 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-eb6445d8-7695-411d-acd3-324dd9b38150 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097382 Aug 07 01:33:09.735616 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 769/1711] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:09 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:10.757289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2fdf484e-f1fb-4b81-b02c-67ebc618d7bd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:10.757945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2fdf484e-f1fb-4b81-b02c-67ebc618d7bd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:10.765150 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fdf484e-f1fb-4b81-b02c-67ebc618d7bd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:10.765647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fdf484e-f1fb-4b81-b02c-67ebc618d7bd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:10.766085 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fdf484e-f1fb-4b81-b02c-67ebc618d7bd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:10.767035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fdf484e-f1fb-4b81-b02c-67ebc618d7bd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:10.767287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fdf484e-f1fb-4b81-b02c-67ebc618d7bd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:10.767710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2fdf484e-f1fb-4b81-b02c-67ebc618d7bd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:10.840760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2fdf484e-f1fb-4b81-b02c-67ebc618d7bd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:10.850734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2fdf484e-f1fb-4b81-b02c-67ebc618d7bd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097240 Aug 07 01:33:10.851344 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 943/1712] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:10 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:11.890782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0d474db0-a090-4fef-a277-4df9c192128d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:11.891958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0d474db0-a090-4fef-a277-4df9c192128d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:11.904152 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d474db0-a090-4fef-a277-4df9c192128d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:11.905566 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d474db0-a090-4fef-a277-4df9c192128d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:11.906355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d474db0-a090-4fef-a277-4df9c192128d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:11.907697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d474db0-a090-4fef-a277-4df9c192128d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:11.908519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d474db0-a090-4fef-a277-4df9c192128d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:11.909481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0d474db0-a090-4fef-a277-4df9c192128d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:12.019864 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0d474db0-a090-4fef-a277-4df9c192128d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:12.033372 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0d474db0-a090-4fef-a277-4df9c192128d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.149976 Aug 07 01:33:12.034424 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 770/1713] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:11 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:13.056366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-78030462-3029-45f0-9405-3213e940ed59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:13.057062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-78030462-3029-45f0-9405-3213e940ed59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:13.064900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78030462-3029-45f0-9405-3213e940ed59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:13.065398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78030462-3029-45f0-9405-3213e940ed59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:13.065844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78030462-3029-45f0-9405-3213e940ed59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:13.066582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78030462-3029-45f0-9405-3213e940ed59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:13.067045 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78030462-3029-45f0-9405-3213e940ed59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:13.067479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78030462-3029-45f0-9405-3213e940ed59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:13.157428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-78030462-3029-45f0-9405-3213e940ed59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:13.166476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-78030462-3029-45f0-9405-3213e940ed59 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.114412 Aug 07 01:33:13.167078 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 944/1714] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:13 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:14.189818 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f6e85570-ff25-49e5-a628-f88df073600c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:14.190447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f6e85570-ff25-49e5-a628-f88df073600c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:14.198089 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6e85570-ff25-49e5-a628-f88df073600c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:14.198617 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6e85570-ff25-49e5-a628-f88df073600c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:14.199041 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6e85570-ff25-49e5-a628-f88df073600c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:14.199919 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6e85570-ff25-49e5-a628-f88df073600c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:14.200435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6e85570-ff25-49e5-a628-f88df073600c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:14.200871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f6e85570-ff25-49e5-a628-f88df073600c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:14.282215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f6e85570-ff25-49e5-a628-f88df073600c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:14.292156 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f6e85570-ff25-49e5-a628-f88df073600c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.107694 Aug 07 01:33:14.293006 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 771/1715] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:14 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:15.314138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0c3a471e-40e2-42aa-964c-b644b6a268b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:15.314739 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0c3a471e-40e2-42aa-964c-b644b6a268b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:15.322393 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c3a471e-40e2-42aa-964c-b644b6a268b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:15.322892 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c3a471e-40e2-42aa-964c-b644b6a268b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:15.323334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c3a471e-40e2-42aa-964c-b644b6a268b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:15.324049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c3a471e-40e2-42aa-964c-b644b6a268b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:15.324553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c3a471e-40e2-42aa-964c-b644b6a268b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:15.324984 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0c3a471e-40e2-42aa-964c-b644b6a268b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:15.393184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0c3a471e-40e2-42aa-964c-b644b6a268b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:15.401960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0c3a471e-40e2-42aa-964c-b644b6a268b4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.091958 Aug 07 01:33:15.402718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 945/1716] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:15 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:16.423702 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d04a81fc-9312-4d0c-ba5b-d761befef34b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:16.424749 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d04a81fc-9312-4d0c-ba5b-d761befef34b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:16.432615 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d04a81fc-9312-4d0c-ba5b-d761befef34b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:16.433104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d04a81fc-9312-4d0c-ba5b-d761befef34b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:16.433546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d04a81fc-9312-4d0c-ba5b-d761befef34b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:16.434267 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d04a81fc-9312-4d0c-ba5b-d761befef34b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:16.434718 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d04a81fc-9312-4d0c-ba5b-d761befef34b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:16.435150 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d04a81fc-9312-4d0c-ba5b-d761befef34b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:16.505345 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d04a81fc-9312-4d0c-ba5b-d761befef34b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:16.514703 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d04a81fc-9312-4d0c-ba5b-d761befef34b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094624 Aug 07 01:33:16.515321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 772/1717] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:16 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:17.538333 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-af58aa7d-d2c9-4282-b01e-c239115b595f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:17.538928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-af58aa7d-d2c9-4282-b01e-c239115b595f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:17.546781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af58aa7d-d2c9-4282-b01e-c239115b595f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:17.547424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af58aa7d-d2c9-4282-b01e-c239115b595f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:17.547921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af58aa7d-d2c9-4282-b01e-c239115b595f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:17.548766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af58aa7d-d2c9-4282-b01e-c239115b595f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:17.549276 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af58aa7d-d2c9-4282-b01e-c239115b595f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:17.549770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-af58aa7d-d2c9-4282-b01e-c239115b595f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:17.647351 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-af58aa7d-d2c9-4282-b01e-c239115b595f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:17.656759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-af58aa7d-d2c9-4282-b01e-c239115b595f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.123050 Aug 07 01:33:17.657522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 946/1718] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:17 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:18.678770 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4c12cfd9-a3a0-46b2-9004-c93a77732380 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:18.679484 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4c12cfd9-a3a0-46b2-9004-c93a77732380 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:18.686943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c12cfd9-a3a0-46b2-9004-c93a77732380 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:18.687443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c12cfd9-a3a0-46b2-9004-c93a77732380 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:18.687881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c12cfd9-a3a0-46b2-9004-c93a77732380 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:18.688657 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c12cfd9-a3a0-46b2-9004-c93a77732380 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:18.689126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c12cfd9-a3a0-46b2-9004-c93a77732380 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:18.689568 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c12cfd9-a3a0-46b2-9004-c93a77732380 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:18.761335 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4c12cfd9-a3a0-46b2-9004-c93a77732380 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:18.776618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4c12cfd9-a3a0-46b2-9004-c93a77732380 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.101426 Aug 07 01:33:18.777676 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 773/1719] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:18 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:19.802616 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f624a7d3-610d-4070-af28-b5ff34ffe8a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:19.803401 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f624a7d3-610d-4070-af28-b5ff34ffe8a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:19.814035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f624a7d3-610d-4070-af28-b5ff34ffe8a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:19.814830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f624a7d3-610d-4070-af28-b5ff34ffe8a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:19.815554 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f624a7d3-610d-4070-af28-b5ff34ffe8a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:19.816735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f624a7d3-610d-4070-af28-b5ff34ffe8a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:19.817476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f624a7d3-610d-4070-af28-b5ff34ffe8a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:19.818196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f624a7d3-610d-4070-af28-b5ff34ffe8a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:19.901657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f624a7d3-610d-4070-af28-b5ff34ffe8a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:19.912429 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f624a7d3-610d-4070-af28-b5ff34ffe8a0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.117195 Aug 07 01:33:19.913871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 947/1720] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:19 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:20.199788 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b73187e2-5549-4192-b0db-cc4d3fb61470 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:20.200694 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b73187e2-5549-4192-b0db-cc4d3fb61470 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:20.211839 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b73187e2-5549-4192-b0db-cc4d3fb61470 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:20.212912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b73187e2-5549-4192-b0db-cc4d3fb61470 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:20.213572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b73187e2-5549-4192-b0db-cc4d3fb61470 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:20.214591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b73187e2-5549-4192-b0db-cc4d3fb61470 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:20.215252 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b73187e2-5549-4192-b0db-cc4d3fb61470 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:20.216007 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b73187e2-5549-4192-b0db-cc4d3fb61470 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:20.272121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b73187e2-5549-4192-b0db-cc4d3fb61470 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Going to try to terminate instance {{(pid=117984) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:33:20.285872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8e8883d0-91a2-4b7b-952d-8455b86fd291 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:20.286395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8e8883d0-91a2-4b7b-952d-8455b86fd291 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:20.296987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e8883d0-91a2-4b7b-952d-8455b86fd291 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:20.297483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e8883d0-91a2-4b7b-952d-8455b86fd291 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:20.298276 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e8883d0-91a2-4b7b-952d-8455b86fd291 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:20.299260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e8883d0-91a2-4b7b-952d-8455b86fd291 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:20.299965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e8883d0-91a2-4b7b-952d-8455b86fd291 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:20.300766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e8883d0-91a2-4b7b-952d-8455b86fd291 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:20.343893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8e8883d0-91a2-4b7b-952d-8455b86fd291 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Going to try to terminate instance {{(pid=117983) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:33:20.509444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b73187e2-5549-4192-b0db-cc4d3fb61470 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 204 len: 0 microversion: 2.1 time: 0.317385 Aug 07 01:33:20.510548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 774/1721] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:33:20 2022] DELETE /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 07 01:33:20.523679 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1bc60199-cac0-42e8-96ed-ba878fd6fbc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:20.525136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1bc60199-cac0-42e8-96ed-ba878fd6fbc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:20.526123 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8e8883d0-91a2-4b7b-952d-8455b86fd291 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "DELETE /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 204 len: 0 microversion: 2.1 time: 0.246514 Aug 07 01:33:20.527205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 948/1722] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:33:20 2022] DELETE /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 0 bytes in 249 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 07 01:33:20.533774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1bc60199-cac0-42e8-96ed-ba878fd6fbc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:20.534957 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1bc60199-cac0-42e8-96ed-ba878fd6fbc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:20.535446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1bc60199-cac0-42e8-96ed-ba878fd6fbc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:20.536224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1bc60199-cac0-42e8-96ed-ba878fd6fbc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:20.536930 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1bc60199-cac0-42e8-96ed-ba878fd6fbc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:20.537372 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1bc60199-cac0-42e8-96ed-ba878fd6fbc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:20.540104 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9a2ec710-5100-46cc-8745-ff51895ac535 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:20.540991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9a2ec710-5100-46cc-8745-ff51895ac535 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:20.549962 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a2ec710-5100-46cc-8745-ff51895ac535 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:20.550590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a2ec710-5100-46cc-8745-ff51895ac535 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:20.551036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a2ec710-5100-46cc-8745-ff51895ac535 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:20.551971 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a2ec710-5100-46cc-8745-ff51895ac535 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:20.552490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a2ec710-5100-46cc-8745-ff51895ac535 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:20.553115 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a2ec710-5100-46cc-8745-ff51895ac535 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:20.783394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1bc60199-cac0-42e8-96ed-ba878fd6fbc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:20.801561 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1bc60199-cac0-42e8-96ed-ba878fd6fbc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.284200 Aug 07 01:33:20.803329 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 775/1723] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:20 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:20.808455 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a8b2e711-f8fe-4bbf-bbdf-c4ddbf5fffef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] Calling method '.version_select at 0x7f1fad6f5280>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:20.814855 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR oslo.messaging._drivers.impl_rabbit [None req-a8b2e711-f8fe-4bbf-bbdf-c4ddbf5fffef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [69b55aee-c31b-4a95-8518-f1d33bb07651] AMQP server on 10.222.0.26:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Aug 07 01:33:20.847803 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9a2ec710-5100-46cc-8745-ff51895ac535 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:20.856783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9a2ec710-5100-46cc-8745-ff51895ac535 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 200 len: 1680 microversion: 2.1 time: 0.322534 Aug 07 01:33:20.857634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 949/1724] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:20 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 1680 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:20.936735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6db6b96d-d8aa-49cf-b529-b5b57f9cc8c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:20.937269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6db6b96d-d8aa-49cf-b529-b5b57f9cc8c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:20.946529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6db6b96d-d8aa-49cf-b529-b5b57f9cc8c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:20.946529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6db6b96d-d8aa-49cf-b529-b5b57f9cc8c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:20.947029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6db6b96d-d8aa-49cf-b529-b5b57f9cc8c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:20.948275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6db6b96d-d8aa-49cf-b529-b5b57f9cc8c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:20.949008 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6db6b96d-d8aa-49cf-b529-b5b57f9cc8c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:20.949595 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6db6b96d-d8aa-49cf-b529-b5b57f9cc8c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:21.041546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6db6b96d-d8aa-49cf-b529-b5b57f9cc8c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:21.057685 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6db6b96d-d8aa-49cf-b529-b5b57f9cc8c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.125311 Aug 07 01:33:21.058769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 950/1725] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:20 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:21.826917 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [None req-a8b2e711-f8fe-4bbf-bbdf-c4ddbf5fffef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] [69b55aee-c31b-4a95-8518-f1d33bb07651] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 35018. Aug 07 01:33:21.843034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0cb5cf0f-ceab-4042-aa09-cbd42f3ad769 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:21.844153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0cb5cf0f-ceab-4042-aa09-cbd42f3ad769 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:21.856665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cb5cf0f-ceab-4042-aa09-cbd42f3ad769 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:21.859340 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cb5cf0f-ceab-4042-aa09-cbd42f3ad769 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:21.860609 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cb5cf0f-ceab-4042-aa09-cbd42f3ad769 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:21.861839 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a8b2e711-f8fe-4bbf-bbdf-c4ddbf5fffef tempest-TestVolumeBootPattern-1662403440 tempest-TestVolumeBootPattern-1662403440-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1022328377" status: 202 len: 0 microversion: 2.1 time: 1.057511 Aug 07 01:33:21.862845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 776/1726] 10.222.0.26 () {58 vars in 1263 bytes} [Sun Aug 7 01:33:20 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1022328377 => generated 0 bytes in 1059 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:33:21.863515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cb5cf0f-ceab-4042-aa09-cbd42f3ad769 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:21.864788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cb5cf0f-ceab-4042-aa09-cbd42f3ad769 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:21.866023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0cb5cf0f-ceab-4042-aa09-cbd42f3ad769 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:21.893507 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-16dabec4-98d4-43bd-978c-42659e4edd42 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:21.894518 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-16dabec4-98d4-43bd-978c-42659e4edd42 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:21.906155 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16dabec4-98d4-43bd-978c-42659e4edd42 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:21.906771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16dabec4-98d4-43bd-978c-42659e4edd42 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:21.907229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16dabec4-98d4-43bd-978c-42659e4edd42 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:21.908051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16dabec4-98d4-43bd-978c-42659e4edd42 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:21.915797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16dabec4-98d4-43bd-978c-42659e4edd42 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:21.916130 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16dabec4-98d4-43bd-978c-42659e4edd42 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:22.152962 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0cb5cf0f-ceab-4042-aa09-cbd42f3ad769 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:22.171120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0cb5cf0f-ceab-4042-aa09-cbd42f3ad769 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.335715 Aug 07 01:33:22.177028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 951/1727] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:21 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:22.187471 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-910d2b3a-ac8b-4ef4-be09-97f07fe8b84b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:22.188826 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-910d2b3a-ac8b-4ef4-be09-97f07fe8b84b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:22.210253 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-910d2b3a-ac8b-4ef4-be09-97f07fe8b84b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:22.211679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-910d2b3a-ac8b-4ef4-be09-97f07fe8b84b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:22.215606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-910d2b3a-ac8b-4ef4-be09-97f07fe8b84b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:22.220400 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-910d2b3a-ac8b-4ef4-be09-97f07fe8b84b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:22.221352 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-910d2b3a-ac8b-4ef4-be09-97f07fe8b84b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:22.222348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-910d2b3a-ac8b-4ef4-be09-97f07fe8b84b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:22.269984 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:33:22.339850 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-910d2b3a-ac8b-4ef4-be09-97f07fe8b84b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:22.351431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-910d2b3a-ac8b-4ef4-be09-97f07fe8b84b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.172624 Aug 07 01:33:22.352426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 952/1728] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:22 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:22.369464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-16dabec4-98d4-43bd-978c-42659e4edd42 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e65f0f1ba54f483abff788974b1256cb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '35820b4f79774ac49f43fce915f56d6f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:22.382958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-16dabec4-98d4-43bd-978c-42659e4edd42 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 200 len: 1680 microversion: 2.1 time: 0.494266 Aug 07 01:33:22.383964 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 777/1729] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:21 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 1680 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:22.666376 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-b7cc0cfd-9e34-487e-9301-df07a801f8bc req-6b68293b-848b-427e-bec9-e59629bb0d38 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1143cfd8-c86b-4023-84cc-8518db39a1be", "name": "network-vif-deleted", "tag": "65d03541-9d7a-43bd-a6b1-fd6ae8b03113"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:33:22.682043 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-b7cc0cfd-9e34-487e-9301-df07a801f8bc req-6b68293b-848b-427e-bec9-e59629bb0d38 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:22.683131 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-b7cc0cfd-9e34-487e-9301-df07a801f8bc req-6b68293b-848b-427e-bec9-e59629bb0d38 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:22.683909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-b7cc0cfd-9e34-487e-9301-df07a801f8bc req-6b68293b-848b-427e-bec9-e59629bb0d38 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:22.702316 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-b7cc0cfd-9e34-487e-9301-df07a801f8bc req-6b68293b-848b-427e-bec9-e59629bb0d38 service nova] Creating event network-vif-deleted:65d03541-9d7a-43bd-a6b1-fd6ae8b03113 for instance 1143cfd8-c86b-4023-84cc-8518db39a1be on hv-cinder-maste Aug 07 01:33:22.712827 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-b7cc0cfd-9e34-487e-9301-df07a801f8bc req-6b68293b-848b-427e-bec9-e59629bb0d38 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049986 Aug 07 01:33:22.715251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 953/1730] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:33:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:33:23.223365 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2e31d1d9-df61-48be-a5d3-a55999557fd5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:23.225621 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2e31d1d9-df61-48be-a5d3-a55999557fd5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:23.243502 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e31d1d9-df61-48be-a5d3-a55999557fd5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:23.244800 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e31d1d9-df61-48be-a5d3-a55999557fd5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:23.245844 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e31d1d9-df61-48be-a5d3-a55999557fd5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:23.247912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e31d1d9-df61-48be-a5d3-a55999557fd5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:23.250097 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e31d1d9-df61-48be-a5d3-a55999557fd5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:23.253531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e31d1d9-df61-48be-a5d3-a55999557fd5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:23.380520 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fe39e052-0cce-4199-9161-f74325789548 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:23.382267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fe39e052-0cce-4199-9161-f74325789548 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:23.392447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fe39e052-0cce-4199-9161-f74325789548 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:23.393341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fe39e052-0cce-4199-9161-f74325789548 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:23.394260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fe39e052-0cce-4199-9161-f74325789548 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:23.395411 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fe39e052-0cce-4199-9161-f74325789548 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:23.396114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fe39e052-0cce-4199-9161-f74325789548 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:23.396819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fe39e052-0cce-4199-9161-f74325789548 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:23.503030 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2e31d1d9-df61-48be-a5d3-a55999557fd5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:23.514008 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2e31d1d9-df61-48be-a5d3-a55999557fd5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.310524 Aug 07 01:33:23.515491 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 778/1731] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:23 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:23.524906 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b83338b1-7975-4fc5-8cbf-6d87b2f38ce9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:23.526435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b83338b1-7975-4fc5-8cbf-6d87b2f38ce9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:23.539362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b83338b1-7975-4fc5-8cbf-6d87b2f38ce9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:23.540203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b83338b1-7975-4fc5-8cbf-6d87b2f38ce9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:23.541319 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b83338b1-7975-4fc5-8cbf-6d87b2f38ce9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:23.542956 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b83338b1-7975-4fc5-8cbf-6d87b2f38ce9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:23.543803 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b83338b1-7975-4fc5-8cbf-6d87b2f38ce9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:23.544906 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b83338b1-7975-4fc5-8cbf-6d87b2f38ce9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:23.569149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-fe39e052-0cce-4199-9161-f74325789548 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:23.574916 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-b83338b1-7975-4fc5-8cbf-6d87b2f38ce9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] HTTP exception thrown: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b83338b1-7975-4fc5-8cbf-6d87b2f38ce9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Request method failure captured: Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be HTTP/1.1 Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xD021Vt2D0N359UOKX9xymFYpnZ79FQqLbXwvg5UbTRYuTTlR_HceEoHaOg5fvzy7nSB6FZNa7SVaU3q4lxIRVTXMupRK2mSDBaPiXGNUdpidSOmvxEoLqM4Zox-kqkoV9_K0_oxsudsRHb6jwecdh1rLe0262HFBUWOJmPZoVity8 Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:33:23.579921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:33:23.583170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:33:23.583170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:33:23.583170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:33:23.583170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:33:23.583170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:33:23.583170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:33:23.583170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:33:23.583170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: e65f0f1ba54f483abff788974b1256cb Aug 07 01:33:23.583170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:33:23.583170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:33:23.583170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:33:23.583170 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '1143cfd8-c86b-4023-84cc-8518db39a1be'} Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:33:23.584688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:33:23.587244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:33:23.589829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fe39e052-0cce-4199-9161-f74325789548 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.204421 Aug 07 01:33:23.589829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 954/1732] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:23 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:23.592144 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b83338b1-7975-4fc5-8cbf-6d87b2f38ce9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Returning 404 to user: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:33:23.593896 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b83338b1-7975-4fc5-8cbf-6d87b2f38ce9 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 404 len: 111 microversion: 2.1 time: 0.076220 Aug 07 01:33:23.595207 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 779/1733] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:23 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 111 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:33:23.606284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ccdf5737-add2-4d0e-9097-ce60c44b927f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '.version_select at 0x7f1fad4a1940>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:23.639175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ccdf5737-add2-4d0e-9097-ce60c44b927f tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-928595154" status: 202 len: 0 microversion: 2.1 time: 0.036733 Aug 07 01:33:23.640092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 955/1734] 10.222.0.26 () {58 vars in 1218 bytes} [Sun Aug 7 01:33:23 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-928595154 => generated 0 bytes in 39 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:33:24.556787 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9e008ea2-8f39-4f9f-87ea-b775de3e32dc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:24.558457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9e008ea2-8f39-4f9f-87ea-b775de3e32dc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:24.567552 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9e008ea2-8f39-4f9f-87ea-b775de3e32dc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:24.568327 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9e008ea2-8f39-4f9f-87ea-b775de3e32dc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:24.568822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9e008ea2-8f39-4f9f-87ea-b775de3e32dc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:24.569925 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9e008ea2-8f39-4f9f-87ea-b775de3e32dc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:24.570498 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9e008ea2-8f39-4f9f-87ea-b775de3e32dc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:24.570936 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9e008ea2-8f39-4f9f-87ea-b775de3e32dc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:24.615083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-44cfed1c-214f-40e5-9c58-fb6bfe91ed71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:24.616607 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-44cfed1c-214f-40e5-9c58-fb6bfe91ed71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:24.630849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44cfed1c-214f-40e5-9c58-fb6bfe91ed71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:24.631518 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44cfed1c-214f-40e5-9c58-fb6bfe91ed71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:24.632180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44cfed1c-214f-40e5-9c58-fb6bfe91ed71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:24.633038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44cfed1c-214f-40e5-9c58-fb6bfe91ed71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:24.633519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44cfed1c-214f-40e5-9c58-fb6bfe91ed71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:24.633963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44cfed1c-214f-40e5-9c58-fb6bfe91ed71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:24.783394 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-44cfed1c-214f-40e5-9c58-fb6bfe91ed71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:24.792590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-44cfed1c-214f-40e5-9c58-fb6bfe91ed71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.182275 Aug 07 01:33:24.793368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 956/1735] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:24 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:24.795263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9e008ea2-8f39-4f9f-87ea-b775de3e32dc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:24.811027 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9e008ea2-8f39-4f9f-87ea-b775de3e32dc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.257381 Aug 07 01:33:24.812454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 780/1736] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:24 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:25.817937 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8c1d3ea6-aa24-4d1b-84da-c58adf9d5411 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:25.819008 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8c1d3ea6-aa24-4d1b-84da-c58adf9d5411 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:25.832731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c1d3ea6-aa24-4d1b-84da-c58adf9d5411 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:25.833917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c1d3ea6-aa24-4d1b-84da-c58adf9d5411 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:25.834826 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c1d3ea6-aa24-4d1b-84da-c58adf9d5411 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:25.836205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c1d3ea6-aa24-4d1b-84da-c58adf9d5411 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:25.837161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c1d3ea6-aa24-4d1b-84da-c58adf9d5411 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:25.838034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c1d3ea6-aa24-4d1b-84da-c58adf9d5411 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:25.841547 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-86e3088c-857a-40d5-bf86-83ad5e62e611 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:25.844361 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-86e3088c-857a-40d5-bf86-83ad5e62e611 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:25.859199 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86e3088c-857a-40d5-bf86-83ad5e62e611 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:25.860283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86e3088c-857a-40d5-bf86-83ad5e62e611 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:25.861423 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86e3088c-857a-40d5-bf86-83ad5e62e611 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:25.862523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86e3088c-857a-40d5-bf86-83ad5e62e611 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:25.863420 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86e3088c-857a-40d5-bf86-83ad5e62e611 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:25.864332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86e3088c-857a-40d5-bf86-83ad5e62e611 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:25.983400 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8c1d3ea6-aa24-4d1b-84da-c58adf9d5411 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:25.997712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8c1d3ea6-aa24-4d1b-84da-c58adf9d5411 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.186554 Aug 07 01:33:25.999264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 957/1737] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:25 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:26.113028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-86e3088c-857a-40d5-bf86-83ad5e62e611 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:26.116631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:33:26.127201 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-86e3088c-857a-40d5-bf86-83ad5e62e611 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.294050 Aug 07 01:33:26.128489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 781/1738] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:25 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:27.026093 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dd6228bf-9f0d-4bca-8579-b1f8e10aca30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:27.027229 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dd6228bf-9f0d-4bca-8579-b1f8e10aca30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:27.036495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd6228bf-9f0d-4bca-8579-b1f8e10aca30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:27.037039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd6228bf-9f0d-4bca-8579-b1f8e10aca30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:27.037492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd6228bf-9f0d-4bca-8579-b1f8e10aca30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:27.038241 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd6228bf-9f0d-4bca-8579-b1f8e10aca30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:27.038715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd6228bf-9f0d-4bca-8579-b1f8e10aca30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:27.039333 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd6228bf-9f0d-4bca-8579-b1f8e10aca30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:27.161933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e6a34979-126d-48df-a1c4-00e32000b814 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:27.163396 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e6a34979-126d-48df-a1c4-00e32000b814 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:27.167677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-dd6228bf-9f0d-4bca-8579-b1f8e10aca30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:27.173741 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6a34979-126d-48df-a1c4-00e32000b814 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:27.174740 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6a34979-126d-48df-a1c4-00e32000b814 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:27.175403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6a34979-126d-48df-a1c4-00e32000b814 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:27.176707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6a34979-126d-48df-a1c4-00e32000b814 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:27.177323 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6a34979-126d-48df-a1c4-00e32000b814 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:27.177772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e6a34979-126d-48df-a1c4-00e32000b814 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:27.184851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dd6228bf-9f0d-4bca-8579-b1f8e10aca30 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.163447 Aug 07 01:33:27.185875 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 958/1739] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:27 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:27.421828 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e6a34979-126d-48df-a1c4-00e32000b814 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:27.444697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e6a34979-126d-48df-a1c4-00e32000b814 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.288819 Aug 07 01:33:27.453945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 782/1740] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:27 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:28.222264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-78bb3d70-d840-4da6-ae13-9caac46a6432 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:28.223598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-78bb3d70-d840-4da6-ae13-9caac46a6432 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:28.238597 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78bb3d70-d840-4da6-ae13-9caac46a6432 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:28.242991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78bb3d70-d840-4da6-ae13-9caac46a6432 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:28.243654 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78bb3d70-d840-4da6-ae13-9caac46a6432 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:28.244745 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78bb3d70-d840-4da6-ae13-9caac46a6432 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:28.245373 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78bb3d70-d840-4da6-ae13-9caac46a6432 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:28.245980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78bb3d70-d840-4da6-ae13-9caac46a6432 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:28.345942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-78bb3d70-d840-4da6-ae13-9caac46a6432 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:28.359242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-78bb3d70-d840-4da6-ae13-9caac46a6432 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.154578 Aug 07 01:33:28.360354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 959/1741] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:28 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:28.471565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6b43a877-6a2f-4de3-a412-e9fa93b8a739 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:28.472187 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6b43a877-6a2f-4de3-a412-e9fa93b8a739 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:28.479099 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b43a877-6a2f-4de3-a412-e9fa93b8a739 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:28.479593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b43a877-6a2f-4de3-a412-e9fa93b8a739 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:28.480096 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b43a877-6a2f-4de3-a412-e9fa93b8a739 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:28.480871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b43a877-6a2f-4de3-a412-e9fa93b8a739 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:28.481346 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b43a877-6a2f-4de3-a412-e9fa93b8a739 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:28.481775 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b43a877-6a2f-4de3-a412-e9fa93b8a739 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:28.757914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6b43a877-6a2f-4de3-a412-e9fa93b8a739 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:28.767994 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6b43a877-6a2f-4de3-a412-e9fa93b8a739 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.300407 Aug 07 01:33:28.780266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 783/1742] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:28 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:29.382430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d92bd031-bffe-484f-ba64-f5a202bf8360 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:29.383857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d92bd031-bffe-484f-ba64-f5a202bf8360 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:29.392194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d92bd031-bffe-484f-ba64-f5a202bf8360 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:29.392730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d92bd031-bffe-484f-ba64-f5a202bf8360 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:29.393281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d92bd031-bffe-484f-ba64-f5a202bf8360 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:29.394037 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d92bd031-bffe-484f-ba64-f5a202bf8360 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:29.395008 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d92bd031-bffe-484f-ba64-f5a202bf8360 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:29.395457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d92bd031-bffe-484f-ba64-f5a202bf8360 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:29.490191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d92bd031-bffe-484f-ba64-f5a202bf8360 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:29.506136 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d92bd031-bffe-484f-ba64-f5a202bf8360 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.127459 Aug 07 01:33:29.507151 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 960/1743] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:29 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:29.797931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4a596520-220b-4ccf-a92c-73c3d1a60f50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:29.799216 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4a596520-220b-4ccf-a92c-73c3d1a60f50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:29.809747 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a596520-220b-4ccf-a92c-73c3d1a60f50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:29.810524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a596520-220b-4ccf-a92c-73c3d1a60f50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:29.811280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a596520-220b-4ccf-a92c-73c3d1a60f50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:29.812586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a596520-220b-4ccf-a92c-73c3d1a60f50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:29.813120 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a596520-220b-4ccf-a92c-73c3d1a60f50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:29.813741 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a596520-220b-4ccf-a92c-73c3d1a60f50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:30.051914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4a596520-220b-4ccf-a92c-73c3d1a60f50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:30.062315 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4a596520-220b-4ccf-a92c-73c3d1a60f50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.272539 Aug 07 01:33:30.063190 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 784/1744] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:29 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:30.530558 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9571fef9-0694-403c-bf1a-1f1424c426c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:30.531759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9571fef9-0694-403c-bf1a-1f1424c426c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:30.536597 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=52,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:33:30.542728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9571fef9-0694-403c-bf1a-1f1424c426c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:30.543600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9571fef9-0694-403c-bf1a-1f1424c426c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:30.544222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9571fef9-0694-403c-bf1a-1f1424c426c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:30.546200 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9571fef9-0694-403c-bf1a-1f1424c426c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:30.547081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9571fef9-0694-403c-bf1a-1f1424c426c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:30.547688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9571fef9-0694-403c-bf1a-1f1424c426c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:30.647212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9571fef9-0694-403c-bf1a-1f1424c426c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:30.658248 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9571fef9-0694-403c-bf1a-1f1424c426c5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.133254 Aug 07 01:33:30.658952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 961/1745] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:30 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:31.096176 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-abea663a-86e8-4247-81b2-667e3b55cab9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:31.097235 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-abea663a-86e8-4247-81b2-667e3b55cab9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:31.100810 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=46,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:33:31.109678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-abea663a-86e8-4247-81b2-667e3b55cab9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:31.110303 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-abea663a-86e8-4247-81b2-667e3b55cab9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:31.110754 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-abea663a-86e8-4247-81b2-667e3b55cab9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:31.111500 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-abea663a-86e8-4247-81b2-667e3b55cab9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:31.112200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-abea663a-86e8-4247-81b2-667e3b55cab9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:31.112848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-abea663a-86e8-4247-81b2-667e3b55cab9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:31.367664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-abea663a-86e8-4247-81b2-667e3b55cab9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:31.378971 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-abea663a-86e8-4247-81b2-667e3b55cab9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.287886 Aug 07 01:33:31.379779 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 785/1746] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:31 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:31.697985 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-128ddf1a-def8-46a6-8603-6de5af7c205a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:31.700297 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-128ddf1a-def8-46a6-8603-6de5af7c205a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:31.714067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-128ddf1a-def8-46a6-8603-6de5af7c205a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:31.715320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-128ddf1a-def8-46a6-8603-6de5af7c205a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:31.716587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-128ddf1a-def8-46a6-8603-6de5af7c205a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:31.718542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-128ddf1a-def8-46a6-8603-6de5af7c205a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:31.719731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-128ddf1a-def8-46a6-8603-6de5af7c205a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:31.720844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-128ddf1a-def8-46a6-8603-6de5af7c205a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:31.839202 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-128ddf1a-def8-46a6-8603-6de5af7c205a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:31.860782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-128ddf1a-def8-46a6-8603-6de5af7c205a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.170446 Aug 07 01:33:31.862974 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 962/1747] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:31 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:32.420649 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0646db8b-b4c9-4c34-9610-67492aa5dc41 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:32.421773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0646db8b-b4c9-4c34-9610-67492aa5dc41 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:32.433822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0646db8b-b4c9-4c34-9610-67492aa5dc41 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:32.434685 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0646db8b-b4c9-4c34-9610-67492aa5dc41 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:32.435525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0646db8b-b4c9-4c34-9610-67492aa5dc41 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:32.436909 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0646db8b-b4c9-4c34-9610-67492aa5dc41 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:32.438749 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0646db8b-b4c9-4c34-9610-67492aa5dc41 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:32.439637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0646db8b-b4c9-4c34-9610-67492aa5dc41 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:32.641295 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0646db8b-b4c9-4c34-9610-67492aa5dc41 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:32.655612 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0646db8b-b4c9-4c34-9610-67492aa5dc41 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.243735 Aug 07 01:33:32.656561 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 786/1748] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:32 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:32.889189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-abbf637b-0078-4cef-ac0a-880e47104b07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:32.889809 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-abbf637b-0078-4cef-ac0a-880e47104b07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:32.901610 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-abbf637b-0078-4cef-ac0a-880e47104b07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:32.903421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-abbf637b-0078-4cef-ac0a-880e47104b07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:32.903886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-abbf637b-0078-4cef-ac0a-880e47104b07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:32.905315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-abbf637b-0078-4cef-ac0a-880e47104b07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:32.905965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-abbf637b-0078-4cef-ac0a-880e47104b07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:32.906482 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-abbf637b-0078-4cef-ac0a-880e47104b07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:33.003815 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-abbf637b-0078-4cef-ac0a-880e47104b07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:33.016390 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-abbf637b-0078-4cef-ac0a-880e47104b07 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.132219 Aug 07 01:33:33.017106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 963/1749] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:32 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:33.692061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7726cdcb-bcec-4508-b7eb-46beb9dab8bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:33.693161 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7726cdcb-bcec-4508-b7eb-46beb9dab8bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:33.705976 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7726cdcb-bcec-4508-b7eb-46beb9dab8bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:33.707711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7726cdcb-bcec-4508-b7eb-46beb9dab8bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:33.708435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7726cdcb-bcec-4508-b7eb-46beb9dab8bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:33.710241 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7726cdcb-bcec-4508-b7eb-46beb9dab8bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:33.711059 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7726cdcb-bcec-4508-b7eb-46beb9dab8bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:33.711846 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7726cdcb-bcec-4508-b7eb-46beb9dab8bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:33.992062 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7726cdcb-bcec-4508-b7eb-46beb9dab8bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:34.006652 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7726cdcb-bcec-4508-b7eb-46beb9dab8bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.321521 Aug 07 01:33:34.007434 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 787/1750] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:33 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:34.050796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-583e4de5-c73e-4c55-83a3-74a5be38224c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:34.052266 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-583e4de5-c73e-4c55-83a3-74a5be38224c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:34.062491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-583e4de5-c73e-4c55-83a3-74a5be38224c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:34.063180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-583e4de5-c73e-4c55-83a3-74a5be38224c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:34.063556 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-583e4de5-c73e-4c55-83a3-74a5be38224c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:34.064494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-583e4de5-c73e-4c55-83a3-74a5be38224c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:34.064796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-583e4de5-c73e-4c55-83a3-74a5be38224c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:34.065738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-583e4de5-c73e-4c55-83a3-74a5be38224c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:34.153311 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-583e4de5-c73e-4c55-83a3-74a5be38224c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:34.167944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-583e4de5-c73e-4c55-83a3-74a5be38224c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.122383 Aug 07 01:33:34.169044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 964/1751] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:34 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:35.043602 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9bd916cf-f584-4e86-b49c-5584a91b58a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:35.044275 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9bd916cf-f584-4e86-b49c-5584a91b58a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:35.052697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9bd916cf-f584-4e86-b49c-5584a91b58a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:35.053480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9bd916cf-f584-4e86-b49c-5584a91b58a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:35.053920 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9bd916cf-f584-4e86-b49c-5584a91b58a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:35.054645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9bd916cf-f584-4e86-b49c-5584a91b58a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:35.055101 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9bd916cf-f584-4e86-b49c-5584a91b58a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:35.055529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9bd916cf-f584-4e86-b49c-5584a91b58a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:35.202727 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a27e2b72-8fbe-4f19-86a6-930b5fcce7da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:35.204075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a27e2b72-8fbe-4f19-86a6-930b5fcce7da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:35.215859 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a27e2b72-8fbe-4f19-86a6-930b5fcce7da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:35.216569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a27e2b72-8fbe-4f19-86a6-930b5fcce7da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:35.217205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a27e2b72-8fbe-4f19-86a6-930b5fcce7da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:35.218289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a27e2b72-8fbe-4f19-86a6-930b5fcce7da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:35.218917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a27e2b72-8fbe-4f19-86a6-930b5fcce7da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:35.219421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a27e2b72-8fbe-4f19-86a6-930b5fcce7da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:35.231615 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9bd916cf-f584-4e86-b49c-5584a91b58a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:35.253540 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9bd916cf-f584-4e86-b49c-5584a91b58a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.213764 Aug 07 01:33:35.255936 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 788/1752] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:35 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:35.306355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a27e2b72-8fbe-4f19-86a6-930b5fcce7da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:35.321110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a27e2b72-8fbe-4f19-86a6-930b5fcce7da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.125269 Aug 07 01:33:35.322030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 965/1753] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:35 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:36.289447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9473fa59-3fd5-4789-80ef-90493e7cd6b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:36.290132 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9473fa59-3fd5-4789-80ef-90493e7cd6b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:36.298212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9473fa59-3fd5-4789-80ef-90493e7cd6b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:36.298623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9473fa59-3fd5-4789-80ef-90493e7cd6b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:36.299071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9473fa59-3fd5-4789-80ef-90493e7cd6b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:36.299827 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9473fa59-3fd5-4789-80ef-90493e7cd6b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:36.300326 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9473fa59-3fd5-4789-80ef-90493e7cd6b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:36.300771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9473fa59-3fd5-4789-80ef-90493e7cd6b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:36.353388 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dd8083c3-e460-418e-9d4a-978a54094e3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:36.354243 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dd8083c3-e460-418e-9d4a-978a54094e3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:36.362250 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd8083c3-e460-418e-9d4a-978a54094e3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:36.362756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd8083c3-e460-418e-9d4a-978a54094e3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:36.363234 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd8083c3-e460-418e-9d4a-978a54094e3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:36.363949 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd8083c3-e460-418e-9d4a-978a54094e3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:36.364457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd8083c3-e460-418e-9d4a-978a54094e3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:36.364893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd8083c3-e460-418e-9d4a-978a54094e3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:36.450004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-dd8083c3-e460-418e-9d4a-978a54094e3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:36.460604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dd8083c3-e460-418e-9d4a-978a54094e3a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.111293 Aug 07 01:33:36.461219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 966/1754] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:36 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:36.479043 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9473fa59-3fd5-4789-80ef-90493e7cd6b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:36.489291 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9473fa59-3fd5-4789-80ef-90493e7cd6b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.203892 Aug 07 01:33:36.490049 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 789/1755] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:36 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:37.489857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f8942a86-706f-4af8-be69-cc5119e243ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:37.490551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f8942a86-706f-4af8-be69-cc5119e243ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:37.497577 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8942a86-706f-4af8-be69-cc5119e243ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:37.498657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8942a86-706f-4af8-be69-cc5119e243ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:37.499129 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8942a86-706f-4af8-be69-cc5119e243ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:37.499939 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8942a86-706f-4af8-be69-cc5119e243ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:37.500451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8942a86-706f-4af8-be69-cc5119e243ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:37.500886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8942a86-706f-4af8-be69-cc5119e243ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:37.511117 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4fba1319-4954-4ac8-a96f-58acfb690f51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:37.511948 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4fba1319-4954-4ac8-a96f-58acfb690f51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:37.520070 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4fba1319-4954-4ac8-a96f-58acfb690f51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:37.520673 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4fba1319-4954-4ac8-a96f-58acfb690f51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:37.521163 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4fba1319-4954-4ac8-a96f-58acfb690f51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:37.521953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4fba1319-4954-4ac8-a96f-58acfb690f51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:37.522484 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4fba1319-4954-4ac8-a96f-58acfb690f51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:37.522890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4fba1319-4954-4ac8-a96f-58acfb690f51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:37.581712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f8942a86-706f-4af8-be69-cc5119e243ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:37.590908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f8942a86-706f-4af8-be69-cc5119e243ad tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.104939 Aug 07 01:33:37.592055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 967/1756] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:37 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:37.709909 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4fba1319-4954-4ac8-a96f-58acfb690f51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:37.718900 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4fba1319-4954-4ac8-a96f-58acfb690f51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.211587 Aug 07 01:33:37.719517 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 790/1757] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:37 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:38.615148 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-71d66e3e-5f32-4d41-8b35-47b3b16b9f46 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:38.615992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-71d66e3e-5f32-4d41-8b35-47b3b16b9f46 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:38.623441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71d66e3e-5f32-4d41-8b35-47b3b16b9f46 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:38.623999 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71d66e3e-5f32-4d41-8b35-47b3b16b9f46 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:38.624594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71d66e3e-5f32-4d41-8b35-47b3b16b9f46 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:38.625377 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71d66e3e-5f32-4d41-8b35-47b3b16b9f46 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:38.625887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71d66e3e-5f32-4d41-8b35-47b3b16b9f46 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:38.626366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-71d66e3e-5f32-4d41-8b35-47b3b16b9f46 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:38.705107 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-71d66e3e-5f32-4d41-8b35-47b3b16b9f46 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:38.714390 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-71d66e3e-5f32-4d41-8b35-47b3b16b9f46 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.103606 Aug 07 01:33:38.715054 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 968/1758] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:38 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:38.741528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a72d07ae-0a9c-4171-a0b8-82fddb21eaaa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:38.742254 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a72d07ae-0a9c-4171-a0b8-82fddb21eaaa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:38.749975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a72d07ae-0a9c-4171-a0b8-82fddb21eaaa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:38.750560 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a72d07ae-0a9c-4171-a0b8-82fddb21eaaa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:38.751240 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a72d07ae-0a9c-4171-a0b8-82fddb21eaaa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:38.752208 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a72d07ae-0a9c-4171-a0b8-82fddb21eaaa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:38.752947 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a72d07ae-0a9c-4171-a0b8-82fddb21eaaa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:38.753504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a72d07ae-0a9c-4171-a0b8-82fddb21eaaa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:38.912096 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a72d07ae-0a9c-4171-a0b8-82fddb21eaaa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:38.924330 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a72d07ae-0a9c-4171-a0b8-82fddb21eaaa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.186585 Aug 07 01:33:38.925223 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 791/1759] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:38 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:39.738066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6a426ec4-0286-430d-89fc-18b6050171fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:39.739004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6a426ec4-0286-430d-89fc-18b6050171fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:39.746371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6a426ec4-0286-430d-89fc-18b6050171fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:39.746867 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6a426ec4-0286-430d-89fc-18b6050171fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:39.747307 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6a426ec4-0286-430d-89fc-18b6050171fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:39.748169 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6a426ec4-0286-430d-89fc-18b6050171fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:39.748652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6a426ec4-0286-430d-89fc-18b6050171fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:39.749083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6a426ec4-0286-430d-89fc-18b6050171fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:39.818332 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6a426ec4-0286-430d-89fc-18b6050171fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:39.827574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6a426ec4-0286-430d-89fc-18b6050171fd tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094000 Aug 07 01:33:39.828197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 969/1760] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:39 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:39.958911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c423f461-4d0e-4f20-b0fd-9806439cc5e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:39.959559 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c423f461-4d0e-4f20-b0fd-9806439cc5e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:39.966707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c423f461-4d0e-4f20-b0fd-9806439cc5e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:39.967206 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c423f461-4d0e-4f20-b0fd-9806439cc5e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:39.967644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c423f461-4d0e-4f20-b0fd-9806439cc5e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:39.968419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c423f461-4d0e-4f20-b0fd-9806439cc5e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:39.968875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c423f461-4d0e-4f20-b0fd-9806439cc5e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:39.969303 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c423f461-4d0e-4f20-b0fd-9806439cc5e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:40.127296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c423f461-4d0e-4f20-b0fd-9806439cc5e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:40.137348 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c423f461-4d0e-4f20-b0fd-9806439cc5e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.182644 Aug 07 01:33:40.137979 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 792/1761] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:39 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:40.850304 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f74c4f34-a633-4520-b37f-cec8295e6502 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:40.851054 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f74c4f34-a633-4520-b37f-cec8295e6502 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:40.860450 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f74c4f34-a633-4520-b37f-cec8295e6502 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:40.861753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f74c4f34-a633-4520-b37f-cec8295e6502 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:40.862765 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f74c4f34-a633-4520-b37f-cec8295e6502 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:40.863586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f74c4f34-a633-4520-b37f-cec8295e6502 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:40.864788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f74c4f34-a633-4520-b37f-cec8295e6502 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:40.864788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f74c4f34-a633-4520-b37f-cec8295e6502 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:40.882650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:33:40.940587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f74c4f34-a633-4520-b37f-cec8295e6502 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:40.949139 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f74c4f34-a633-4520-b37f-cec8295e6502 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.102776 Aug 07 01:33:40.949910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 970/1762] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:40 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:41.163711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8feeaa49-1b63-4c9d-9e51-36eb2a0be106 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:41.164376 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8feeaa49-1b63-4c9d-9e51-36eb2a0be106 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:41.171952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8feeaa49-1b63-4c9d-9e51-36eb2a0be106 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:41.172483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8feeaa49-1b63-4c9d-9e51-36eb2a0be106 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:41.172926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8feeaa49-1b63-4c9d-9e51-36eb2a0be106 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:41.173656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8feeaa49-1b63-4c9d-9e51-36eb2a0be106 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:41.174117 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8feeaa49-1b63-4c9d-9e51-36eb2a0be106 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:41.174551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8feeaa49-1b63-4c9d-9e51-36eb2a0be106 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:41.333428 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8feeaa49-1b63-4c9d-9e51-36eb2a0be106 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:41.342289 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8feeaa49-1b63-4c9d-9e51-36eb2a0be106 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.182076 Aug 07 01:33:41.342878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 793/1763] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:41 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:41.972026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-27884f14-ce91-4edc-ad7f-e0927d53863f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:41.972699 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-27884f14-ce91-4edc-ad7f-e0927d53863f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:41.979490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27884f14-ce91-4edc-ad7f-e0927d53863f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:41.979989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27884f14-ce91-4edc-ad7f-e0927d53863f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:41.980463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27884f14-ce91-4edc-ad7f-e0927d53863f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:41.981204 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27884f14-ce91-4edc-ad7f-e0927d53863f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:41.981659 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27884f14-ce91-4edc-ad7f-e0927d53863f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:41.982587 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27884f14-ce91-4edc-ad7f-e0927d53863f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:42.057984 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-27884f14-ce91-4edc-ad7f-e0927d53863f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:42.067973 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-27884f14-ce91-4edc-ad7f-e0927d53863f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.099792 Aug 07 01:33:42.068612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 971/1764] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:41 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:42.363931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a389c3c1-1a37-4969-bebb-8a0da03769be tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:42.364611 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a389c3c1-1a37-4969-bebb-8a0da03769be tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:42.371743 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a389c3c1-1a37-4969-bebb-8a0da03769be tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:42.372329 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a389c3c1-1a37-4969-bebb-8a0da03769be tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:42.372998 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a389c3c1-1a37-4969-bebb-8a0da03769be tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:42.373817 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a389c3c1-1a37-4969-bebb-8a0da03769be tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:42.374323 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a389c3c1-1a37-4969-bebb-8a0da03769be tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:42.374793 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a389c3c1-1a37-4969-bebb-8a0da03769be tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:42.549844 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a389c3c1-1a37-4969-bebb-8a0da03769be tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:42.558584 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a389c3c1-1a37-4969-bebb-8a0da03769be tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.198523 Aug 07 01:33:42.559186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 794/1765] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:42 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:43.089672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-17ed27fb-945b-4f0f-b509-dc45e3935e50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:43.090353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-17ed27fb-945b-4f0f-b509-dc45e3935e50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:43.098806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17ed27fb-945b-4f0f-b509-dc45e3935e50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:43.099415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17ed27fb-945b-4f0f-b509-dc45e3935e50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:43.099869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17ed27fb-945b-4f0f-b509-dc45e3935e50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:43.100754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17ed27fb-945b-4f0f-b509-dc45e3935e50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:43.101224 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17ed27fb-945b-4f0f-b509-dc45e3935e50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:43.101823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-17ed27fb-945b-4f0f-b509-dc45e3935e50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:43.182283 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-17ed27fb-945b-4f0f-b509-dc45e3935e50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:43.191421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-17ed27fb-945b-4f0f-b509-dc45e3935e50 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.105620 Aug 07 01:33:43.192014 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 972/1766] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:43 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:43.581031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2b180b60-61ba-4bff-91e1-b98fdca2a94b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:43.582028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2b180b60-61ba-4bff-91e1-b98fdca2a94b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:43.589101 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2b180b60-61ba-4bff-91e1-b98fdca2a94b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:43.589627 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2b180b60-61ba-4bff-91e1-b98fdca2a94b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:43.590100 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2b180b60-61ba-4bff-91e1-b98fdca2a94b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:43.590872 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2b180b60-61ba-4bff-91e1-b98fdca2a94b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:43.591998 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2b180b60-61ba-4bff-91e1-b98fdca2a94b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:43.592528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2b180b60-61ba-4bff-91e1-b98fdca2a94b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:43.746419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2b180b60-61ba-4bff-91e1-b98fdca2a94b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:43.755571 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2b180b60-61ba-4bff-91e1-b98fdca2a94b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.178706 Aug 07 01:33:43.756169 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 795/1767] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:43 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:44.213068 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-16ee455d-06f6-4047-a148-df478b68e125 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:44.213680 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-16ee455d-06f6-4047-a148-df478b68e125 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:44.221036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-16ee455d-06f6-4047-a148-df478b68e125 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:44.221528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-16ee455d-06f6-4047-a148-df478b68e125 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:44.221972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-16ee455d-06f6-4047-a148-df478b68e125 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:44.222685 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-16ee455d-06f6-4047-a148-df478b68e125 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:44.223137 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-16ee455d-06f6-4047-a148-df478b68e125 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:44.223559 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-16ee455d-06f6-4047-a148-df478b68e125 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:44.311726 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-16ee455d-06f6-4047-a148-df478b68e125 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:44.320716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-16ee455d-06f6-4047-a148-df478b68e125 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.111436 Aug 07 01:33:44.321320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 973/1768] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:44 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:44.778118 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3164bac8-2c91-40f7-9a57-5052d3c6b969 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:44.779603 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3164bac8-2c91-40f7-9a57-5052d3c6b969 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:44.786353 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3164bac8-2c91-40f7-9a57-5052d3c6b969 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:44.786853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3164bac8-2c91-40f7-9a57-5052d3c6b969 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:44.787660 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3164bac8-2c91-40f7-9a57-5052d3c6b969 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:44.788427 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3164bac8-2c91-40f7-9a57-5052d3c6b969 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:44.788888 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3164bac8-2c91-40f7-9a57-5052d3c6b969 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:44.789311 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3164bac8-2c91-40f7-9a57-5052d3c6b969 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:44.943567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3164bac8-2c91-40f7-9a57-5052d3c6b969 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:44.953650 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3164bac8-2c91-40f7-9a57-5052d3c6b969 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.179452 Aug 07 01:33:44.954251 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 796/1769] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:44 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:45.342944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d4f8bc39-62ad-400b-baaa-f63fc14975c2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:45.343595 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d4f8bc39-62ad-400b-baaa-f63fc14975c2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:45.350456 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4f8bc39-62ad-400b-baaa-f63fc14975c2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:45.350975 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4f8bc39-62ad-400b-baaa-f63fc14975c2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:45.351417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4f8bc39-62ad-400b-baaa-f63fc14975c2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:45.352137 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4f8bc39-62ad-400b-baaa-f63fc14975c2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:45.352861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4f8bc39-62ad-400b-baaa-f63fc14975c2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:45.353296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d4f8bc39-62ad-400b-baaa-f63fc14975c2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:45.427641 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d4f8bc39-62ad-400b-baaa-f63fc14975c2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:45.436815 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d4f8bc39-62ad-400b-baaa-f63fc14975c2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097734 Aug 07 01:33:45.437418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 974/1770] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:45 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:45.978611 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-260347d8-2062-4f93-b8bb-05f727b03d78 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:45.979248 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-260347d8-2062-4f93-b8bb-05f727b03d78 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:45.987348 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-260347d8-2062-4f93-b8bb-05f727b03d78 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:45.987846 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-260347d8-2062-4f93-b8bb-05f727b03d78 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:45.988299 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-260347d8-2062-4f93-b8bb-05f727b03d78 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:45.989016 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-260347d8-2062-4f93-b8bb-05f727b03d78 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:45.989466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-260347d8-2062-4f93-b8bb-05f727b03d78 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:45.989890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-260347d8-2062-4f93-b8bb-05f727b03d78 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:46.151260 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-260347d8-2062-4f93-b8bb-05f727b03d78 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:46.160555 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-260347d8-2062-4f93-b8bb-05f727b03d78 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.186281 Aug 07 01:33:46.161153 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 797/1771] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:45 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:46.464664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-058a2ec1-39c4-4888-821d-140e8a542d8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:46.465353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-058a2ec1-39c4-4888-821d-140e8a542d8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:46.473492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-058a2ec1-39c4-4888-821d-140e8a542d8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:46.473987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-058a2ec1-39c4-4888-821d-140e8a542d8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:46.474434 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-058a2ec1-39c4-4888-821d-140e8a542d8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:46.475152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-058a2ec1-39c4-4888-821d-140e8a542d8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:46.475614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-058a2ec1-39c4-4888-821d-140e8a542d8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:46.476047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-058a2ec1-39c4-4888-821d-140e8a542d8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:46.542574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-058a2ec1-39c4-4888-821d-140e8a542d8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:46.551346 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-058a2ec1-39c4-4888-821d-140e8a542d8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.090588 Aug 07 01:33:46.551947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 975/1772] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:46 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:47.183007 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ff716cb9-ae4f-412b-a2dc-307cdb8df6af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:47.183698 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ff716cb9-ae4f-412b-a2dc-307cdb8df6af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:47.191403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff716cb9-ae4f-412b-a2dc-307cdb8df6af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:47.191892 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff716cb9-ae4f-412b-a2dc-307cdb8df6af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:47.192366 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff716cb9-ae4f-412b-a2dc-307cdb8df6af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:47.193083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff716cb9-ae4f-412b-a2dc-307cdb8df6af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:47.193531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff716cb9-ae4f-412b-a2dc-307cdb8df6af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:47.193957 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ff716cb9-ae4f-412b-a2dc-307cdb8df6af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:47.210633 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:33:47.345059 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ff716cb9-ae4f-412b-a2dc-307cdb8df6af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:47.353546 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ff716cb9-ae4f-412b-a2dc-307cdb8df6af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.174437 Aug 07 01:33:47.354140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 798/1773] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:47 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:47.573628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e86298d6-e1e5-4950-b118-468ff444fbfe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:47.574382 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e86298d6-e1e5-4950-b118-468ff444fbfe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:47.581561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e86298d6-e1e5-4950-b118-468ff444fbfe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:47.582061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e86298d6-e1e5-4950-b118-468ff444fbfe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:47.582511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e86298d6-e1e5-4950-b118-468ff444fbfe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:47.583237 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e86298d6-e1e5-4950-b118-468ff444fbfe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:47.583703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e86298d6-e1e5-4950-b118-468ff444fbfe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:47.584139 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e86298d6-e1e5-4950-b118-468ff444fbfe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:47.657136 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e86298d6-e1e5-4950-b118-468ff444fbfe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:47.665997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e86298d6-e1e5-4950-b118-468ff444fbfe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.096785 Aug 07 01:33:47.666600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 976/1774] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:47 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:48.379213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d9ae5abe-acd4-4ad2-abea-db1ee21386d6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:48.380390 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d9ae5abe-acd4-4ad2-abea-db1ee21386d6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:48.395209 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9ae5abe-acd4-4ad2-abea-db1ee21386d6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:48.395704 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9ae5abe-acd4-4ad2-abea-db1ee21386d6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:48.396149 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9ae5abe-acd4-4ad2-abea-db1ee21386d6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:48.396933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9ae5abe-acd4-4ad2-abea-db1ee21386d6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:48.397482 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9ae5abe-acd4-4ad2-abea-db1ee21386d6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:48.398063 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d9ae5abe-acd4-4ad2-abea-db1ee21386d6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:48.434826 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:33:48.435176 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:33:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c (ip 10.222.0.26) !!! Aug 07 01:33:48.615103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d9ae5abe-acd4-4ad2-abea-db1ee21386d6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:48.627568 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d9ae5abe-acd4-4ad2-abea-db1ee21386d6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.255867 Aug 07 01:33:48.628175 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 799/1775] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:48 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:48.687770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cc40d8e1-5e0e-4385-a15b-e9d871067923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:48.688450 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cc40d8e1-5e0e-4385-a15b-e9d871067923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:48.695424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cc40d8e1-5e0e-4385-a15b-e9d871067923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:48.695925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cc40d8e1-5e0e-4385-a15b-e9d871067923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:48.696409 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cc40d8e1-5e0e-4385-a15b-e9d871067923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:48.697135 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cc40d8e1-5e0e-4385-a15b-e9d871067923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:48.697599 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cc40d8e1-5e0e-4385-a15b-e9d871067923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:48.698034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cc40d8e1-5e0e-4385-a15b-e9d871067923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:48.782335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-cc40d8e1-5e0e-4385-a15b-e9d871067923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:48.796742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cc40d8e1-5e0e-4385-a15b-e9d871067923 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.113329 Aug 07 01:33:48.797387 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 977/1776] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:48 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:49.649706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-960bdae6-3a3b-479a-a778-5259b786782e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:49.650306 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-960bdae6-3a3b-479a-a778-5259b786782e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:49.657366 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-960bdae6-3a3b-479a-a778-5259b786782e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:49.657866 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-960bdae6-3a3b-479a-a778-5259b786782e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:49.658310 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-960bdae6-3a3b-479a-a778-5259b786782e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:49.659048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-960bdae6-3a3b-479a-a778-5259b786782e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:49.659506 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-960bdae6-3a3b-479a-a778-5259b786782e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:49.659933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-960bdae6-3a3b-479a-a778-5259b786782e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:49.819615 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-960bdae6-3a3b-479a-a778-5259b786782e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:49.827779 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dc3bd1f9-e0de-4302-97b6-016b614687e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:49.828446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-960bdae6-3a3b-479a-a778-5259b786782e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.182487 Aug 07 01:33:49.828874 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 800/1777] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:49 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:49.829083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dc3bd1f9-e0de-4302-97b6-016b614687e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:49.839993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc3bd1f9-e0de-4302-97b6-016b614687e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:49.840926 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc3bd1f9-e0de-4302-97b6-016b614687e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:49.842337 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc3bd1f9-e0de-4302-97b6-016b614687e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:49.843590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc3bd1f9-e0de-4302-97b6-016b614687e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:49.844468 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc3bd1f9-e0de-4302-97b6-016b614687e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:49.845232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dc3bd1f9-e0de-4302-97b6-016b614687e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:49.917194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-dc3bd1f9-e0de-4302-97b6-016b614687e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:49.926384 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dc3bd1f9-e0de-4302-97b6-016b614687e3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.106147 Aug 07 01:33:49.926989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 978/1778] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:49 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:50.856193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-562d6592-7f54-4d68-940d-40c879f0655a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:50.857713 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-562d6592-7f54-4d68-940d-40c879f0655a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:50.865521 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-562d6592-7f54-4d68-940d-40c879f0655a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:50.866016 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-562d6592-7f54-4d68-940d-40c879f0655a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:50.866463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-562d6592-7f54-4d68-940d-40c879f0655a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:50.867172 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-562d6592-7f54-4d68-940d-40c879f0655a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:50.867637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-562d6592-7f54-4d68-940d-40c879f0655a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:50.868070 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-562d6592-7f54-4d68-940d-40c879f0655a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:50.947716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f804997a-f08f-464d-8c83-915ed70cca8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:50.949706 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f804997a-f08f-464d-8c83-915ed70cca8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:50.958546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f804997a-f08f-464d-8c83-915ed70cca8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:50.959405 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f804997a-f08f-464d-8c83-915ed70cca8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:50.960178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f804997a-f08f-464d-8c83-915ed70cca8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:50.961426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f804997a-f08f-464d-8c83-915ed70cca8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:50.962050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f804997a-f08f-464d-8c83-915ed70cca8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:50.963224 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f804997a-f08f-464d-8c83-915ed70cca8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:51.043535 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-562d6592-7f54-4d68-940d-40c879f0655a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:51.060926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-562d6592-7f54-4d68-940d-40c879f0655a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.212077 Aug 07 01:33:51.076321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 801/1779] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:50 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:51.077422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f804997a-f08f-464d-8c83-915ed70cca8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:51.082819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f804997a-f08f-464d-8c83-915ed70cca8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.139200 Aug 07 01:33:51.083856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 979/1780] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:50 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:52.093861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5eabf78f-3521-4c40-9d53-b4eb00f7df6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:52.094764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5eabf78f-3521-4c40-9d53-b4eb00f7df6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:52.101926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5eabf78f-3521-4c40-9d53-b4eb00f7df6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:52.102797 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5eabf78f-3521-4c40-9d53-b4eb00f7df6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:52.103503 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5eabf78f-3521-4c40-9d53-b4eb00f7df6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:52.104495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5eabf78f-3521-4c40-9d53-b4eb00f7df6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:52.105189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5eabf78f-3521-4c40-9d53-b4eb00f7df6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:52.105724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5eabf78f-3521-4c40-9d53-b4eb00f7df6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:52.109180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c89dcab4-1a4f-43d6-b8e1-08a70c48fa8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:52.110137 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c89dcab4-1a4f-43d6-b8e1-08a70c48fa8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:52.118555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c89dcab4-1a4f-43d6-b8e1-08a70c48fa8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:52.119139 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c89dcab4-1a4f-43d6-b8e1-08a70c48fa8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:52.119656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c89dcab4-1a4f-43d6-b8e1-08a70c48fa8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:52.120941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c89dcab4-1a4f-43d6-b8e1-08a70c48fa8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:52.121495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c89dcab4-1a4f-43d6-b8e1-08a70c48fa8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:52.122016 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c89dcab4-1a4f-43d6-b8e1-08a70c48fa8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:52.226437 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c89dcab4-1a4f-43d6-b8e1-08a70c48fa8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:52.235605 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c89dcab4-1a4f-43d6-b8e1-08a70c48fa8d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.131646 Aug 07 01:33:52.236214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 980/1781] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:52 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:52.302214 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5eabf78f-3521-4c40-9d53-b4eb00f7df6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:52.311231 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5eabf78f-3521-4c40-9d53-b4eb00f7df6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.223280 Aug 07 01:33:52.311911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 802/1782] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:52 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:53.257369 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ec644a5f-a607-4cee-b0b0-57b607f6087f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:53.258170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ec644a5f-a607-4cee-b0b0-57b607f6087f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:53.266828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec644a5f-a607-4cee-b0b0-57b607f6087f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:53.267593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec644a5f-a607-4cee-b0b0-57b607f6087f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:53.268389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec644a5f-a607-4cee-b0b0-57b607f6087f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:53.269499 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec644a5f-a607-4cee-b0b0-57b607f6087f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:53.270174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec644a5f-a607-4cee-b0b0-57b607f6087f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:53.270658 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec644a5f-a607-4cee-b0b0-57b607f6087f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:53.336001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3b7969b2-bb2d-42b9-9abe-7fb24b2d307e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:53.337031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3b7969b2-bb2d-42b9-9abe-7fb24b2d307e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:53.347730 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b7969b2-bb2d-42b9-9abe-7fb24b2d307e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:53.348499 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b7969b2-bb2d-42b9-9abe-7fb24b2d307e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:53.349200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b7969b2-bb2d-42b9-9abe-7fb24b2d307e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:53.350366 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b7969b2-bb2d-42b9-9abe-7fb24b2d307e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:53.351116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b7969b2-bb2d-42b9-9abe-7fb24b2d307e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:53.351799 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b7969b2-bb2d-42b9-9abe-7fb24b2d307e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:53.573193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3b7969b2-bb2d-42b9-9abe-7fb24b2d307e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:53.590081 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3b7969b2-bb2d-42b9-9abe-7fb24b2d307e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.260237 Aug 07 01:33:53.591588 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 803/1783] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:53 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:53.639088 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ec644a5f-a607-4cee-b0b0-57b607f6087f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:53.654576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ec644a5f-a607-4cee-b0b0-57b607f6087f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.400175 Aug 07 01:33:53.655655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 981/1784] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:53 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:54.625474 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2f0c8670-1ae4-4ab7-87b6-2cf651a2369f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:54.626481 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2f0c8670-1ae4-4ab7-87b6-2cf651a2369f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:54.633717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f0c8670-1ae4-4ab7-87b6-2cf651a2369f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:54.634208 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f0c8670-1ae4-4ab7-87b6-2cf651a2369f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:54.634648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f0c8670-1ae4-4ab7-87b6-2cf651a2369f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:54.635357 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f0c8670-1ae4-4ab7-87b6-2cf651a2369f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:54.635809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f0c8670-1ae4-4ab7-87b6-2cf651a2369f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:54.636225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2f0c8670-1ae4-4ab7-87b6-2cf651a2369f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:54.692415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-080255c1-8c86-44b0-bc9a-a148512cb9c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:54.693433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-080255c1-8c86-44b0-bc9a-a148512cb9c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:54.701488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-080255c1-8c86-44b0-bc9a-a148512cb9c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:54.702228 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-080255c1-8c86-44b0-bc9a-a148512cb9c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:54.702778 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-080255c1-8c86-44b0-bc9a-a148512cb9c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:54.703609 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-080255c1-8c86-44b0-bc9a-a148512cb9c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:54.704166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-080255c1-8c86-44b0-bc9a-a148512cb9c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:54.704730 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-080255c1-8c86-44b0-bc9a-a148512cb9c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:54.782818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-080255c1-8c86-44b0-bc9a-a148512cb9c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:54.791623 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-080255c1-8c86-44b0-bc9a-a148512cb9c4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.103519 Aug 07 01:33:54.792214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 982/1785] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:54 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:54.816387 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2f0c8670-1ae4-4ab7-87b6-2cf651a2369f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:54.826630 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2f0c8670-1ae4-4ab7-87b6-2cf651a2369f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.206017 Aug 07 01:33:54.827225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 804/1786] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:54 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:55.815663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-aed249e4-2595-43c8-b62e-46d212187f49 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:55.816459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-aed249e4-2595-43c8-b62e-46d212187f49 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:55.823075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aed249e4-2595-43c8-b62e-46d212187f49 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:55.823571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aed249e4-2595-43c8-b62e-46d212187f49 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:55.824013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aed249e4-2595-43c8-b62e-46d212187f49 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:55.824786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aed249e4-2595-43c8-b62e-46d212187f49 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:55.825245 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aed249e4-2595-43c8-b62e-46d212187f49 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:55.826291 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aed249e4-2595-43c8-b62e-46d212187f49 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:55.852139 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e098efc8-2f46-4383-bb9b-8319f653577a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:55.853321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e098efc8-2f46-4383-bb9b-8319f653577a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:55.861255 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e098efc8-2f46-4383-bb9b-8319f653577a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:55.861848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e098efc8-2f46-4383-bb9b-8319f653577a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:55.862326 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e098efc8-2f46-4383-bb9b-8319f653577a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:55.863128 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e098efc8-2f46-4383-bb9b-8319f653577a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:55.863620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e098efc8-2f46-4383-bb9b-8319f653577a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:55.864312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e098efc8-2f46-4383-bb9b-8319f653577a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:55.916110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-aed249e4-2595-43c8-b62e-46d212187f49 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:55.929551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-aed249e4-2595-43c8-b62e-46d212187f49 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.117371 Aug 07 01:33:55.930654 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 983/1787] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:55 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:56.041721 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e098efc8-2f46-4383-bb9b-8319f653577a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:56.051706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e098efc8-2f46-4383-bb9b-8319f653577a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.204445 Aug 07 01:33:56.052351 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 805/1788] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:55 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:33:56.953368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9d5c8567-b6a0-4db6-9571-e93c43906bba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:56.954119 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9d5c8567-b6a0-4db6-9571-e93c43906bba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:56.961956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d5c8567-b6a0-4db6-9571-e93c43906bba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:56.962509 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d5c8567-b6a0-4db6-9571-e93c43906bba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:56.962986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d5c8567-b6a0-4db6-9571-e93c43906bba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:56.963743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d5c8567-b6a0-4db6-9571-e93c43906bba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:56.964379 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d5c8567-b6a0-4db6-9571-e93c43906bba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:56.964864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d5c8567-b6a0-4db6-9571-e93c43906bba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:57.032321 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9d5c8567-b6a0-4db6-9571-e93c43906bba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:57.041287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9d5c8567-b6a0-4db6-9571-e93c43906bba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.091932 Aug 07 01:33:57.041889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 984/1789] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:56 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:57.077849 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ecc484fe-26be-4e83-b009-bbf6e6dd0e65 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:57.078505 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ecc484fe-26be-4e83-b009-bbf6e6dd0e65 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:57.085942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecc484fe-26be-4e83-b009-bbf6e6dd0e65 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:57.086482 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecc484fe-26be-4e83-b009-bbf6e6dd0e65 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:57.086954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecc484fe-26be-4e83-b009-bbf6e6dd0e65 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:57.087845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecc484fe-26be-4e83-b009-bbf6e6dd0e65 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:57.088512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecc484fe-26be-4e83-b009-bbf6e6dd0e65 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:57.089033 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ecc484fe-26be-4e83-b009-bbf6e6dd0e65 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:57.262828 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ecc484fe-26be-4e83-b009-bbf6e6dd0e65 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:57.271710 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ecc484fe-26be-4e83-b009-bbf6e6dd0e65 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.197900 Aug 07 01:33:57.272510 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 806/1790] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:57 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:58.066260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6c76d8f8-d90c-4e81-a71b-e4bb7b3caebc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:58.066983 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6c76d8f8-d90c-4e81-a71b-e4bb7b3caebc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:58.074026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c76d8f8-d90c-4e81-a71b-e4bb7b3caebc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:58.074524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c76d8f8-d90c-4e81-a71b-e4bb7b3caebc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:58.074955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c76d8f8-d90c-4e81-a71b-e4bb7b3caebc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:58.076296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c76d8f8-d90c-4e81-a71b-e4bb7b3caebc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:58.076756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c76d8f8-d90c-4e81-a71b-e4bb7b3caebc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:58.077184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c76d8f8-d90c-4e81-a71b-e4bb7b3caebc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:58.157891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6c76d8f8-d90c-4e81-a71b-e4bb7b3caebc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:58.166540 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6c76d8f8-d90c-4e81-a71b-e4bb7b3caebc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.105000 Aug 07 01:33:58.167140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 985/1791] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:58 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:58.295583 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ded4ba21-abd8-4e99-a60d-92a11b865d87 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:58.296298 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ded4ba21-abd8-4e99-a60d-92a11b865d87 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:58.303552 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ded4ba21-abd8-4e99-a60d-92a11b865d87 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:58.304043 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ded4ba21-abd8-4e99-a60d-92a11b865d87 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:58.304516 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ded4ba21-abd8-4e99-a60d-92a11b865d87 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:58.305249 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ded4ba21-abd8-4e99-a60d-92a11b865d87 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:58.305706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ded4ba21-abd8-4e99-a60d-92a11b865d87 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:58.306138 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ded4ba21-abd8-4e99-a60d-92a11b865d87 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:58.458499 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ded4ba21-abd8-4e99-a60d-92a11b865d87 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:58.473838 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ded4ba21-abd8-4e99-a60d-92a11b865d87 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.182479 Aug 07 01:33:58.474910 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 807/1792] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:58 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:59.192209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6622ce54-922d-4373-a095-2149d1f33d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:59.193368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6622ce54-922d-4373-a095-2149d1f33d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:59.204686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6622ce54-922d-4373-a095-2149d1f33d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:59.205565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6622ce54-922d-4373-a095-2149d1f33d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:59.206616 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6622ce54-922d-4373-a095-2149d1f33d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:59.207913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6622ce54-922d-4373-a095-2149d1f33d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:59.208760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6622ce54-922d-4373-a095-2149d1f33d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:59.209558 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6622ce54-922d-4373-a095-2149d1f33d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:59.282578 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6622ce54-922d-4373-a095-2149d1f33d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:59.297307 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6622ce54-922d-4373-a095-2149d1f33d0b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.113309 Aug 07 01:33:59.298356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 986/1793] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:59 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:33:59.511120 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a39c251a-d26d-4a86-b641-6dd4d1ceddd1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:33:59.511762 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a39c251a-d26d-4a86-b641-6dd4d1ceddd1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:33:59.518766 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a39c251a-d26d-4a86-b641-6dd4d1ceddd1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:59.519294 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a39c251a-d26d-4a86-b641-6dd4d1ceddd1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:59.519778 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a39c251a-d26d-4a86-b641-6dd4d1ceddd1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:59.520590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a39c251a-d26d-4a86-b641-6dd4d1ceddd1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:33:59.521101 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a39c251a-d26d-4a86-b641-6dd4d1ceddd1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:33:59.521565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a39c251a-d26d-4a86-b641-6dd4d1ceddd1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:33:59.670220 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a39c251a-d26d-4a86-b641-6dd4d1ceddd1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:33:59.678761 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a39c251a-d26d-4a86-b641-6dd4d1ceddd1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.171616 Aug 07 01:33:59.679358 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 808/1794] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:33:59 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:00.332496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2ad64e43-43a0-46af-93f3-44ba7d51f821 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:00.333110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2ad64e43-43a0-46af-93f3-44ba7d51f821 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:00.339861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2ad64e43-43a0-46af-93f3-44ba7d51f821 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:00.340401 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2ad64e43-43a0-46af-93f3-44ba7d51f821 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:00.340847 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2ad64e43-43a0-46af-93f3-44ba7d51f821 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:00.342075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2ad64e43-43a0-46af-93f3-44ba7d51f821 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:00.342541 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2ad64e43-43a0-46af-93f3-44ba7d51f821 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:00.342978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2ad64e43-43a0-46af-93f3-44ba7d51f821 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:00.346036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:34:00.411979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2ad64e43-43a0-46af-93f3-44ba7d51f821 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:00.422200 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2ad64e43-43a0-46af-93f3-44ba7d51f821 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094038 Aug 07 01:34:00.422976 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 987/1795] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:00 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:00.701018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2e48bd50-cf55-4f19-a151-08fcaadac0a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:00.701615 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2e48bd50-cf55-4f19-a151-08fcaadac0a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:00.709533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e48bd50-cf55-4f19-a151-08fcaadac0a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:00.710021 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e48bd50-cf55-4f19-a151-08fcaadac0a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:00.710458 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e48bd50-cf55-4f19-a151-08fcaadac0a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:00.711184 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e48bd50-cf55-4f19-a151-08fcaadac0a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:00.711660 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e48bd50-cf55-4f19-a151-08fcaadac0a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:00.712081 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2e48bd50-cf55-4f19-a151-08fcaadac0a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:01.160862 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2e48bd50-cf55-4f19-a151-08fcaadac0a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:01.170430 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2e48bd50-cf55-4f19-a151-08fcaadac0a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.474006 Aug 07 01:34:01.171093 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 809/1796] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:00 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:01.445381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dd093da2-515d-455c-886e-d28ba4338b24 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:01.446287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dd093da2-515d-455c-886e-d28ba4338b24 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:01.449112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=54 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:34:01.456340 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd093da2-515d-455c-886e-d28ba4338b24 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:01.456839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd093da2-515d-455c-886e-d28ba4338b24 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:01.457282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd093da2-515d-455c-886e-d28ba4338b24 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:01.458007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd093da2-515d-455c-886e-d28ba4338b24 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:01.458464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd093da2-515d-455c-886e-d28ba4338b24 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:01.458896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dd093da2-515d-455c-886e-d28ba4338b24 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:01.529155 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-dd093da2-515d-455c-886e-d28ba4338b24 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:01.540706 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dd093da2-515d-455c-886e-d28ba4338b24 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.100328 Aug 07 01:34:01.541320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 988/1797] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:01 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:02.192193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e933e0eb-4115-438b-a14c-d9faae62ea7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:02.193002 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e933e0eb-4115-438b-a14c-d9faae62ea7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:02.195238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=50 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:34:02.201422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e933e0eb-4115-438b-a14c-d9faae62ea7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:02.201972 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e933e0eb-4115-438b-a14c-d9faae62ea7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:02.202483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e933e0eb-4115-438b-a14c-d9faae62ea7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:02.203393 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e933e0eb-4115-438b-a14c-d9faae62ea7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:02.204051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e933e0eb-4115-438b-a14c-d9faae62ea7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:02.204617 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e933e0eb-4115-438b-a14c-d9faae62ea7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:02.347980 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e933e0eb-4115-438b-a14c-d9faae62ea7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:02.356943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e933e0eb-4115-438b-a14c-d9faae62ea7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.168569 Aug 07 01:34:02.357543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 810/1798] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:02 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:02.563028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-98dde633-5bf3-40c0-8ba3-7edf6e66e1a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:02.563657 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-98dde633-5bf3-40c0-8ba3-7edf6e66e1a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:02.570990 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98dde633-5bf3-40c0-8ba3-7edf6e66e1a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:02.571516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98dde633-5bf3-40c0-8ba3-7edf6e66e1a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:02.571958 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98dde633-5bf3-40c0-8ba3-7edf6e66e1a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:02.572734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98dde633-5bf3-40c0-8ba3-7edf6e66e1a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:02.573190 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98dde633-5bf3-40c0-8ba3-7edf6e66e1a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:02.573618 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-98dde633-5bf3-40c0-8ba3-7edf6e66e1a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:02.668576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-98dde633-5bf3-40c0-8ba3-7edf6e66e1a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:02.677596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-98dde633-5bf3-40c0-8ba3-7edf6e66e1a4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.118857 Aug 07 01:34:02.678305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 989/1799] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:02 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:03.379567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8fea5839-0168-4335-bd68-d6b84827f13a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:03.380269 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8fea5839-0168-4335-bd68-d6b84827f13a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:03.387189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8fea5839-0168-4335-bd68-d6b84827f13a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:03.387713 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8fea5839-0168-4335-bd68-d6b84827f13a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:03.388750 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8fea5839-0168-4335-bd68-d6b84827f13a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:03.389485 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8fea5839-0168-4335-bd68-d6b84827f13a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:03.389946 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8fea5839-0168-4335-bd68-d6b84827f13a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:03.390379 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8fea5839-0168-4335-bd68-d6b84827f13a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:03.540575 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8fea5839-0168-4335-bd68-d6b84827f13a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:03.552117 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8fea5839-0168-4335-bd68-d6b84827f13a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.176264 Aug 07 01:34:03.552999 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 811/1800] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:03 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:03.704598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6af4557c-6cb5-41d2-b1be-c2fb5792befe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:03.705683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6af4557c-6cb5-41d2-b1be-c2fb5792befe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:03.717349 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6af4557c-6cb5-41d2-b1be-c2fb5792befe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:03.718205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6af4557c-6cb5-41d2-b1be-c2fb5792befe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:03.718989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6af4557c-6cb5-41d2-b1be-c2fb5792befe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:03.720326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6af4557c-6cb5-41d2-b1be-c2fb5792befe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:03.721159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6af4557c-6cb5-41d2-b1be-c2fb5792befe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:03.721941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6af4557c-6cb5-41d2-b1be-c2fb5792befe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:03.793186 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6af4557c-6cb5-41d2-b1be-c2fb5792befe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:03.802898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6af4557c-6cb5-41d2-b1be-c2fb5792befe tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.105966 Aug 07 01:34:03.803498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 990/1801] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:03 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:04.577468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-61208c8c-817c-497c-a75c-74863245a1ac tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:04.578513 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-61208c8c-817c-497c-a75c-74863245a1ac tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:04.586122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61208c8c-817c-497c-a75c-74863245a1ac tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:04.586749 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61208c8c-817c-497c-a75c-74863245a1ac tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:04.587250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61208c8c-817c-497c-a75c-74863245a1ac tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:04.588099 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61208c8c-817c-497c-a75c-74863245a1ac tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:04.588707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61208c8c-817c-497c-a75c-74863245a1ac tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:04.589204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61208c8c-817c-497c-a75c-74863245a1ac tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:04.774876 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-61208c8c-817c-497c-a75c-74863245a1ac tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:04.784174 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-61208c8c-817c-497c-a75c-74863245a1ac tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.211553 Aug 07 01:34:04.784814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 812/1802] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:04 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:04.825819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-db313754-03f5-41e7-975d-7ae98166f4e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:04.826615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-db313754-03f5-41e7-975d-7ae98166f4e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:04.833902 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-db313754-03f5-41e7-975d-7ae98166f4e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:04.834321 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-db313754-03f5-41e7-975d-7ae98166f4e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:04.834799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-db313754-03f5-41e7-975d-7ae98166f4e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:04.835574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-db313754-03f5-41e7-975d-7ae98166f4e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:04.836081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-db313754-03f5-41e7-975d-7ae98166f4e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:04.836546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-db313754-03f5-41e7-975d-7ae98166f4e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:04.906584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-db313754-03f5-41e7-975d-7ae98166f4e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:04.920173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-db313754-03f5-41e7-975d-7ae98166f4e1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098038 Aug 07 01:34:04.921293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 991/1803] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:04 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:05.959917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c8994165-f894-49a7-a4c6-09434ab3bbf8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:05.961009 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c8994165-f894-49a7-a4c6-09434ab3bbf8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:05.968661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8994165-f894-49a7-a4c6-09434ab3bbf8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:05.969162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8994165-f894-49a7-a4c6-09434ab3bbf8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:05.969599 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8994165-f894-49a7-a4c6-09434ab3bbf8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:05.970407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8994165-f894-49a7-a4c6-09434ab3bbf8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:05.970881 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8994165-f894-49a7-a4c6-09434ab3bbf8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:05.971318 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8994165-f894-49a7-a4c6-09434ab3bbf8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:06.055659 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c8994165-f894-49a7-a4c6-09434ab3bbf8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:06.067745 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c8994165-f894-49a7-a4c6-09434ab3bbf8 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.117327 Aug 07 01:34:06.070455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 992/1804] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:05 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:06.135459 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-94bb49e6-8763-4aba-b948-75092d5fde28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:06.136337 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-94bb49e6-8763-4aba-b948-75092d5fde28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:06.143338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-94bb49e6-8763-4aba-b948-75092d5fde28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:06.143880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-94bb49e6-8763-4aba-b948-75092d5fde28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:06.144417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-94bb49e6-8763-4aba-b948-75092d5fde28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:06.145229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-94bb49e6-8763-4aba-b948-75092d5fde28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:06.145808 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-94bb49e6-8763-4aba-b948-75092d5fde28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:06.146282 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-94bb49e6-8763-4aba-b948-75092d5fde28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:06.316690 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-94bb49e6-8763-4aba-b948-75092d5fde28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:06.325884 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-94bb49e6-8763-4aba-b948-75092d5fde28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.522815 Aug 07 01:34:06.326493 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 813/1805] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:05 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:07.093623 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f79f37fc-81c1-4e0a-be2e-f50412d97339 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:07.094405 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f79f37fc-81c1-4e0a-be2e-f50412d97339 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:07.102743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f79f37fc-81c1-4e0a-be2e-f50412d97339 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:07.103334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f79f37fc-81c1-4e0a-be2e-f50412d97339 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:07.103831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f79f37fc-81c1-4e0a-be2e-f50412d97339 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:07.104702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f79f37fc-81c1-4e0a-be2e-f50412d97339 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:07.105224 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f79f37fc-81c1-4e0a-be2e-f50412d97339 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:07.105931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f79f37fc-81c1-4e0a-be2e-f50412d97339 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:07.185939 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f79f37fc-81c1-4e0a-be2e-f50412d97339 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:07.195832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f79f37fc-81c1-4e0a-be2e-f50412d97339 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.107036 Aug 07 01:34:07.196485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 993/1806] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:07 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:07.347963 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ca375dfb-d1ae-49dd-98c5-9b4cd45b7b9a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:07.348675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ca375dfb-d1ae-49dd-98c5-9b4cd45b7b9a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:07.355661 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca375dfb-d1ae-49dd-98c5-9b4cd45b7b9a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:07.356156 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca375dfb-d1ae-49dd-98c5-9b4cd45b7b9a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:07.356724 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca375dfb-d1ae-49dd-98c5-9b4cd45b7b9a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:07.357497 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca375dfb-d1ae-49dd-98c5-9b4cd45b7b9a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:07.358123 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca375dfb-d1ae-49dd-98c5-9b4cd45b7b9a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:07.358557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ca375dfb-d1ae-49dd-98c5-9b4cd45b7b9a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:07.507864 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ca375dfb-d1ae-49dd-98c5-9b4cd45b7b9a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:07.517307 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ca375dfb-d1ae-49dd-98c5-9b4cd45b7b9a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.173459 Aug 07 01:34:07.517913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 814/1807] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:07 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:08.218186 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7bffae4c-5b6d-4c3e-af05-4f46dbfba4b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:08.218815 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7bffae4c-5b6d-4c3e-af05-4f46dbfba4b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:08.225918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bffae4c-5b6d-4c3e-af05-4f46dbfba4b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:08.226414 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bffae4c-5b6d-4c3e-af05-4f46dbfba4b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:08.226860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bffae4c-5b6d-4c3e-af05-4f46dbfba4b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:08.227586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bffae4c-5b6d-4c3e-af05-4f46dbfba4b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:08.228046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bffae4c-5b6d-4c3e-af05-4f46dbfba4b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:08.228514 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7bffae4c-5b6d-4c3e-af05-4f46dbfba4b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:08.319081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7bffae4c-5b6d-4c3e-af05-4f46dbfba4b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:08.328551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7bffae4c-5b6d-4c3e-af05-4f46dbfba4b7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.114312 Aug 07 01:34:08.329151 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 994/1808] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:08 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:08.539488 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5921d410-66f3-4690-a182-da6db370e64c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:08.540174 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5921d410-66f3-4690-a182-da6db370e64c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:08.548088 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5921d410-66f3-4690-a182-da6db370e64c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:08.548606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5921d410-66f3-4690-a182-da6db370e64c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:08.549050 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5921d410-66f3-4690-a182-da6db370e64c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:08.549771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5921d410-66f3-4690-a182-da6db370e64c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:08.550237 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5921d410-66f3-4690-a182-da6db370e64c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:08.550667 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5921d410-66f3-4690-a182-da6db370e64c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:08.553812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:34:08.698832 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5921d410-66f3-4690-a182-da6db370e64c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:08.707842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5921d410-66f3-4690-a182-da6db370e64c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.172329 Aug 07 01:34:08.708460 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 815/1809] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:08 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:09.350565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7244f8ff-8df4-4b77-ba11-58bdf7ef5d58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:09.351228 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7244f8ff-8df4-4b77-ba11-58bdf7ef5d58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:09.358920 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7244f8ff-8df4-4b77-ba11-58bdf7ef5d58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:09.359415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7244f8ff-8df4-4b77-ba11-58bdf7ef5d58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:09.359856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7244f8ff-8df4-4b77-ba11-58bdf7ef5d58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:09.360617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7244f8ff-8df4-4b77-ba11-58bdf7ef5d58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:09.361113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7244f8ff-8df4-4b77-ba11-58bdf7ef5d58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:09.361542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7244f8ff-8df4-4b77-ba11-58bdf7ef5d58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:09.446497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7244f8ff-8df4-4b77-ba11-58bdf7ef5d58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:09.455174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7244f8ff-8df4-4b77-ba11-58bdf7ef5d58 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.108515 Aug 07 01:34:09.455785 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 995/1810] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:09 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:09.742905 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1fd30671-ece0-4749-8736-fbc6cf5d1795 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:09.743613 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1fd30671-ece0-4749-8736-fbc6cf5d1795 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:09.751032 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1fd30671-ece0-4749-8736-fbc6cf5d1795 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:09.751510 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1fd30671-ece0-4749-8736-fbc6cf5d1795 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:09.751955 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1fd30671-ece0-4749-8736-fbc6cf5d1795 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:09.752721 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1fd30671-ece0-4749-8736-fbc6cf5d1795 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:09.753179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1fd30671-ece0-4749-8736-fbc6cf5d1795 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:09.753628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1fd30671-ece0-4749-8736-fbc6cf5d1795 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:09.925829 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1fd30671-ece0-4749-8736-fbc6cf5d1795 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:09.934430 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1fd30671-ece0-4749-8736-fbc6cf5d1795 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.195533 Aug 07 01:34:09.935645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 816/1811] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:09 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:10.476992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6203f877-2544-46d1-bd5c-1e764e5cdc0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:10.477610 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6203f877-2544-46d1-bd5c-1e764e5cdc0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:10.484774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6203f877-2544-46d1-bd5c-1e764e5cdc0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:10.485274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6203f877-2544-46d1-bd5c-1e764e5cdc0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:10.485716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6203f877-2544-46d1-bd5c-1e764e5cdc0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:10.486445 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6203f877-2544-46d1-bd5c-1e764e5cdc0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:10.486908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6203f877-2544-46d1-bd5c-1e764e5cdc0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:10.487379 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6203f877-2544-46d1-bd5c-1e764e5cdc0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:10.559958 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6203f877-2544-46d1-bd5c-1e764e5cdc0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:10.574074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6203f877-2544-46d1-bd5c-1e764e5cdc0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.100803 Aug 07 01:34:10.574955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 996/1812] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:10 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:10.971245 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f0c7909e-65c6-4ed4-baf5-45949da66355 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:10.971935 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f0c7909e-65c6-4ed4-baf5-45949da66355 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:10.980702 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0c7909e-65c6-4ed4-baf5-45949da66355 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:10.981264 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0c7909e-65c6-4ed4-baf5-45949da66355 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:10.981716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0c7909e-65c6-4ed4-baf5-45949da66355 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:10.982534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0c7909e-65c6-4ed4-baf5-45949da66355 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:10.983011 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0c7909e-65c6-4ed4-baf5-45949da66355 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:10.983464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0c7909e-65c6-4ed4-baf5-45949da66355 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:11.142829 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f0c7909e-65c6-4ed4-baf5-45949da66355 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:11.152212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f0c7909e-65c6-4ed4-baf5-45949da66355 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.184849 Aug 07 01:34:11.152864 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 817/1813] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:10 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:11.596995 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-dcd40df9-21f9-4a63-9f8d-7a306b64ef18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:11.597561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-dcd40df9-21f9-4a63-9f8d-7a306b64ef18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:11.604227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dcd40df9-21f9-4a63-9f8d-7a306b64ef18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:11.604762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dcd40df9-21f9-4a63-9f8d-7a306b64ef18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:11.605209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dcd40df9-21f9-4a63-9f8d-7a306b64ef18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:11.605937 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dcd40df9-21f9-4a63-9f8d-7a306b64ef18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:11.606959 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dcd40df9-21f9-4a63-9f8d-7a306b64ef18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:11.607396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-dcd40df9-21f9-4a63-9f8d-7a306b64ef18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:11.677232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-dcd40df9-21f9-4a63-9f8d-7a306b64ef18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:11.686492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-dcd40df9-21f9-4a63-9f8d-7a306b64ef18 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.093470 Aug 07 01:34:11.687084 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 997/1814] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:11 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:12.176298 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8d068fa4-07ca-4c47-95b6-e91c4f0aa70f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:12.176907 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8d068fa4-07ca-4c47-95b6-e91c4f0aa70f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:12.183722 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d068fa4-07ca-4c47-95b6-e91c4f0aa70f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:12.184215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d068fa4-07ca-4c47-95b6-e91c4f0aa70f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:12.185265 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d068fa4-07ca-4c47-95b6-e91c4f0aa70f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:12.186003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d068fa4-07ca-4c47-95b6-e91c4f0aa70f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:12.186458 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d068fa4-07ca-4c47-95b6-e91c4f0aa70f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:12.186887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d068fa4-07ca-4c47-95b6-e91c4f0aa70f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:12.380622 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8d068fa4-07ca-4c47-95b6-e91c4f0aa70f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:12.389171 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8d068fa4-07ca-4c47-95b6-e91c4f0aa70f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.217100 Aug 07 01:34:12.389776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 818/1815] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:12 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:12.707965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4881a16b-ce96-47b6-b1ea-ea157085c53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:12.708679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4881a16b-ce96-47b6-b1ea-ea157085c53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:12.715566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4881a16b-ce96-47b6-b1ea-ea157085c53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:12.716103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4881a16b-ce96-47b6-b1ea-ea157085c53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:12.716592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4881a16b-ce96-47b6-b1ea-ea157085c53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:12.717322 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4881a16b-ce96-47b6-b1ea-ea157085c53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:12.717782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4881a16b-ce96-47b6-b1ea-ea157085c53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:12.718212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4881a16b-ce96-47b6-b1ea-ea157085c53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:12.788604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4881a16b-ce96-47b6-b1ea-ea157085c53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:12.804219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4881a16b-ce96-47b6-b1ea-ea157085c53a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.100047 Aug 07 01:34:12.805330 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 998/1816] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:12 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:13.410917 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e669b2be-3216-409d-ae6b-257974a18c26 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:13.411557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e669b2be-3216-409d-ae6b-257974a18c26 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:13.418434 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e669b2be-3216-409d-ae6b-257974a18c26 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:13.419538 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e669b2be-3216-409d-ae6b-257974a18c26 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:13.419987 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e669b2be-3216-409d-ae6b-257974a18c26 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:13.420736 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e669b2be-3216-409d-ae6b-257974a18c26 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:13.421184 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e669b2be-3216-409d-ae6b-257974a18c26 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:13.421606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e669b2be-3216-409d-ae6b-257974a18c26 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:13.579760 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e669b2be-3216-409d-ae6b-257974a18c26 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:13.589109 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e669b2be-3216-409d-ae6b-257974a18c26 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.182230 Aug 07 01:34:13.589716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 819/1817] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:13 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:13.828010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4b5a746f-a91e-4fb5-affe-9cfdc31add0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:13.828761 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4b5a746f-a91e-4fb5-affe-9cfdc31add0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:13.835820 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b5a746f-a91e-4fb5-affe-9cfdc31add0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:13.836343 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b5a746f-a91e-4fb5-affe-9cfdc31add0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:13.836796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b5a746f-a91e-4fb5-affe-9cfdc31add0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:13.837510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b5a746f-a91e-4fb5-affe-9cfdc31add0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:13.837967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b5a746f-a91e-4fb5-affe-9cfdc31add0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:13.838393 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4b5a746f-a91e-4fb5-affe-9cfdc31add0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:13.909218 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4b5a746f-a91e-4fb5-affe-9cfdc31add0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:13.917634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4b5a746f-a91e-4fb5-affe-9cfdc31add0a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094137 Aug 07 01:34:13.918313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 999/1818] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:13 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:14.614514 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-86e5c1a2-923f-4361-9d07-8e1415388962 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:14.615320 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-86e5c1a2-923f-4361-9d07-8e1415388962 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:14.625558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86e5c1a2-923f-4361-9d07-8e1415388962 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:14.626154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86e5c1a2-923f-4361-9d07-8e1415388962 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:14.626597 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86e5c1a2-923f-4361-9d07-8e1415388962 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:14.627450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86e5c1a2-923f-4361-9d07-8e1415388962 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:14.628228 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86e5c1a2-923f-4361-9d07-8e1415388962 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:14.628814 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-86e5c1a2-923f-4361-9d07-8e1415388962 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:14.777539 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-86e5c1a2-923f-4361-9d07-8e1415388962 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:14.786451 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-86e5c1a2-923f-4361-9d07-8e1415388962 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.178480 Aug 07 01:34:14.787150 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 820/1819] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:14 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:14.939579 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-81e6a16e-2bb8-48d3-8e1a-a7a52fe18f5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:14.940303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-81e6a16e-2bb8-48d3-8e1a-a7a52fe18f5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:14.947145 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81e6a16e-2bb8-48d3-8e1a-a7a52fe18f5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:14.947636 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81e6a16e-2bb8-48d3-8e1a-a7a52fe18f5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:14.948073 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81e6a16e-2bb8-48d3-8e1a-a7a52fe18f5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:14.948827 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81e6a16e-2bb8-48d3-8e1a-a7a52fe18f5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:14.949281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81e6a16e-2bb8-48d3-8e1a-a7a52fe18f5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:14.949737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81e6a16e-2bb8-48d3-8e1a-a7a52fe18f5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:15.017762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-81e6a16e-2bb8-48d3-8e1a-a7a52fe18f5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:15.026444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-81e6a16e-2bb8-48d3-8e1a-a7a52fe18f5b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.091257 Aug 07 01:34:15.027038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1000/1820] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:14 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:15.808753 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c12b234a-1b05-47d0-9da7-ab3aabf2f85a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:15.809982 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c12b234a-1b05-47d0-9da7-ab3aabf2f85a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:15.817064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c12b234a-1b05-47d0-9da7-ab3aabf2f85a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:15.817567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c12b234a-1b05-47d0-9da7-ab3aabf2f85a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:15.818003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c12b234a-1b05-47d0-9da7-ab3aabf2f85a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:15.818728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c12b234a-1b05-47d0-9da7-ab3aabf2f85a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:15.819179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c12b234a-1b05-47d0-9da7-ab3aabf2f85a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:15.819598 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c12b234a-1b05-47d0-9da7-ab3aabf2f85a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:15.973608 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c12b234a-1b05-47d0-9da7-ab3aabf2f85a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:15.988190 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c12b234a-1b05-47d0-9da7-ab3aabf2f85a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.183059 Aug 07 01:34:15.989348 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 821/1821] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:15 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:16.048762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8259c0ac-2886-44da-9ef9-3a20d52ddda2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:16.049465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8259c0ac-2886-44da-9ef9-3a20d52ddda2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:16.056400 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8259c0ac-2886-44da-9ef9-3a20d52ddda2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:16.056953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8259c0ac-2886-44da-9ef9-3a20d52ddda2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:16.057556 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8259c0ac-2886-44da-9ef9-3a20d52ddda2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:16.058417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8259c0ac-2886-44da-9ef9-3a20d52ddda2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:16.059023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8259c0ac-2886-44da-9ef9-3a20d52ddda2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:16.061802 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8259c0ac-2886-44da-9ef9-3a20d52ddda2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:16.134447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8259c0ac-2886-44da-9ef9-3a20d52ddda2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:16.143090 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8259c0ac-2886-44da-9ef9-3a20d52ddda2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098834 Aug 07 01:34:16.143700 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1001/1822] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:16 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:17.024346 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-304a66ee-14a6-4db9-bad3-ef109fa71f96 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:17.025147 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-304a66ee-14a6-4db9-bad3-ef109fa71f96 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:17.032642 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-304a66ee-14a6-4db9-bad3-ef109fa71f96 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:17.033131 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-304a66ee-14a6-4db9-bad3-ef109fa71f96 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:17.033566 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-304a66ee-14a6-4db9-bad3-ef109fa71f96 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:17.034286 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-304a66ee-14a6-4db9-bad3-ef109fa71f96 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:17.034737 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-304a66ee-14a6-4db9-bad3-ef109fa71f96 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:17.035160 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-304a66ee-14a6-4db9-bad3-ef109fa71f96 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:17.178206 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-01899348-1f47-45d9-afc1-b33ce85adb40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:17.179177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-01899348-1f47-45d9-afc1-b33ce85adb40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:17.180373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-304a66ee-14a6-4db9-bad3-ef109fa71f96 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:17.188566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-01899348-1f47-45d9-afc1-b33ce85adb40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:17.189221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-01899348-1f47-45d9-afc1-b33ce85adb40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:17.189633 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-304a66ee-14a6-4db9-bad3-ef109fa71f96 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.169981 Aug 07 01:34:17.189980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-01899348-1f47-45d9-afc1-b33ce85adb40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:17.190180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 822/1823] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:17 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:17.191304 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-01899348-1f47-45d9-afc1-b33ce85adb40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:17.191960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-01899348-1f47-45d9-afc1-b33ce85adb40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:17.192598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-01899348-1f47-45d9-afc1-b33ce85adb40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:17.261800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-01899348-1f47-45d9-afc1-b33ce85adb40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:17.270347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-01899348-1f47-45d9-afc1-b33ce85adb40 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.099398 Aug 07 01:34:17.270940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1002/1824] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:17 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:18.215602 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e2362ae1-c362-4c7d-af02-4ceef88c445d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:18.216707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e2362ae1-c362-4c7d-af02-4ceef88c445d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:18.228190 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2362ae1-c362-4c7d-af02-4ceef88c445d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:18.229083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2362ae1-c362-4c7d-af02-4ceef88c445d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:18.229858 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2362ae1-c362-4c7d-af02-4ceef88c445d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:18.231114 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2362ae1-c362-4c7d-af02-4ceef88c445d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:18.232460 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2362ae1-c362-4c7d-af02-4ceef88c445d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:18.233224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e2362ae1-c362-4c7d-af02-4ceef88c445d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:18.292822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4790453d-7767-4169-8cf3-4f82d972bb0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:18.294394 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4790453d-7767-4169-8cf3-4f82d972bb0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:18.302591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4790453d-7767-4169-8cf3-4f82d972bb0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:18.303315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4790453d-7767-4169-8cf3-4f82d972bb0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:18.303985 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4790453d-7767-4169-8cf3-4f82d972bb0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:18.305543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4790453d-7767-4169-8cf3-4f82d972bb0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:18.306388 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4790453d-7767-4169-8cf3-4f82d972bb0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:18.307012 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4790453d-7767-4169-8cf3-4f82d972bb0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:18.407124 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e2362ae1-c362-4c7d-af02-4ceef88c445d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:18.407399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4790453d-7767-4169-8cf3-4f82d972bb0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:18.410438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:34:18.418154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e2362ae1-c362-4c7d-af02-4ceef88c445d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.209774 Aug 07 01:34:18.419437 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 823/1825] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:18 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:18.419986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4790453d-7767-4169-8cf3-4f82d972bb0e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.131306 Aug 07 01:34:18.420646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1003/1826] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:18 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:19.442856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c09ed463-975a-4ee8-9d4d-1309f13f4ec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:19.443708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c09ed463-975a-4ee8-9d4d-1309f13f4ec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:19.446102 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5b63a417-4dfd-4a9d-b7bd-388142f3dd64 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:19.447241 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5b63a417-4dfd-4a9d-b7bd-388142f3dd64 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:19.453855 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c09ed463-975a-4ee8-9d4d-1309f13f4ec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:19.454582 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c09ed463-975a-4ee8-9d4d-1309f13f4ec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:19.455148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c09ed463-975a-4ee8-9d4d-1309f13f4ec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:19.455836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b63a417-4dfd-4a9d-b7bd-388142f3dd64 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:19.456054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c09ed463-975a-4ee8-9d4d-1309f13f4ec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:19.456750 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b63a417-4dfd-4a9d-b7bd-388142f3dd64 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:19.457018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c09ed463-975a-4ee8-9d4d-1309f13f4ec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:19.457396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b63a417-4dfd-4a9d-b7bd-388142f3dd64 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:19.457641 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c09ed463-975a-4ee8-9d4d-1309f13f4ec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:19.458238 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b63a417-4dfd-4a9d-b7bd-388142f3dd64 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:19.458766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b63a417-4dfd-4a9d-b7bd-388142f3dd64 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:19.459279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b63a417-4dfd-4a9d-b7bd-388142f3dd64 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:19.550393 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c09ed463-975a-4ee8-9d4d-1309f13f4ec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:19.565342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c09ed463-975a-4ee8-9d4d-1309f13f4ec1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.127466 Aug 07 01:34:19.566672 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 824/1827] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:19 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:19.666202 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5b63a417-4dfd-4a9d-b7bd-388142f3dd64 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:19.675187 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5b63a417-4dfd-4a9d-b7bd-388142f3dd64 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.236613 Aug 07 01:34:19.675873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1004/1828] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:19 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:20.595951 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-15c387cd-fb69-48cc-82ee-409e20a3800d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:20.596633 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-15c387cd-fb69-48cc-82ee-409e20a3800d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:20.604065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15c387cd-fb69-48cc-82ee-409e20a3800d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:20.604745 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15c387cd-fb69-48cc-82ee-409e20a3800d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:20.605284 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15c387cd-fb69-48cc-82ee-409e20a3800d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:20.606182 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15c387cd-fb69-48cc-82ee-409e20a3800d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:20.606733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15c387cd-fb69-48cc-82ee-409e20a3800d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:20.607256 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-15c387cd-fb69-48cc-82ee-409e20a3800d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:20.675902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-15c387cd-fb69-48cc-82ee-409e20a3800d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:20.684730 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-15c387cd-fb69-48cc-82ee-409e20a3800d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.092676 Aug 07 01:34:20.685336 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 825/1829] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:20 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:20.700652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ef8f8750-f63b-4362-b214-9e107029d321 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:20.701563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ef8f8750-f63b-4362-b214-9e107029d321 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:20.709167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef8f8750-f63b-4362-b214-9e107029d321 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:20.709673 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef8f8750-f63b-4362-b214-9e107029d321 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:20.710120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef8f8750-f63b-4362-b214-9e107029d321 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:20.710847 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef8f8750-f63b-4362-b214-9e107029d321 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:20.711310 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef8f8750-f63b-4362-b214-9e107029d321 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:20.712076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef8f8750-f63b-4362-b214-9e107029d321 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:20.889966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ef8f8750-f63b-4362-b214-9e107029d321 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:20.898737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ef8f8750-f63b-4362-b214-9e107029d321 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.203053 Aug 07 01:34:20.899335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1005/1830] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:20 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:21.714490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cc6d14ea-96ed-4b5a-a685-c54d6509414e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:21.715471 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-cc6d14ea-96ed-4b5a-a685-c54d6509414e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:21.726798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc6d14ea-96ed-4b5a-a685-c54d6509414e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:21.727632 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc6d14ea-96ed-4b5a-a685-c54d6509414e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:21.728436 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc6d14ea-96ed-4b5a-a685-c54d6509414e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:21.729702 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc6d14ea-96ed-4b5a-a685-c54d6509414e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:21.730543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc6d14ea-96ed-4b5a-a685-c54d6509414e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:21.731305 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cc6d14ea-96ed-4b5a-a685-c54d6509414e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:21.818048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-cc6d14ea-96ed-4b5a-a685-c54d6509414e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:21.827971 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-cc6d14ea-96ed-4b5a-a685-c54d6509414e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.120767 Aug 07 01:34:21.828596 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 826/1831] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:21 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:21.924061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-31e8b3a0-f5a2-47f7-a869-6d988a87b69c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:21.924862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-31e8b3a0-f5a2-47f7-a869-6d988a87b69c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:21.932081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31e8b3a0-f5a2-47f7-a869-6d988a87b69c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:21.932622 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31e8b3a0-f5a2-47f7-a869-6d988a87b69c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:21.933062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31e8b3a0-f5a2-47f7-a869-6d988a87b69c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:21.933784 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31e8b3a0-f5a2-47f7-a869-6d988a87b69c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:21.934234 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31e8b3a0-f5a2-47f7-a869-6d988a87b69c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:21.934655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-31e8b3a0-f5a2-47f7-a869-6d988a87b69c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:22.093968 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-31e8b3a0-f5a2-47f7-a869-6d988a87b69c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:22.106596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-31e8b3a0-f5a2-47f7-a869-6d988a87b69c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.187637 Aug 07 01:34:22.107629 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1006/1832] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:21 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:22.850238 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-470c3b45-ae78-4bce-8fb9-25ca6f9270da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:22.850941 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-470c3b45-ae78-4bce-8fb9-25ca6f9270da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:22.858425 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-470c3b45-ae78-4bce-8fb9-25ca6f9270da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:22.858912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-470c3b45-ae78-4bce-8fb9-25ca6f9270da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:22.859342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-470c3b45-ae78-4bce-8fb9-25ca6f9270da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:22.860048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-470c3b45-ae78-4bce-8fb9-25ca6f9270da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:22.860534 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-470c3b45-ae78-4bce-8fb9-25ca6f9270da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:22.860957 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-470c3b45-ae78-4bce-8fb9-25ca6f9270da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:22.934015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-470c3b45-ae78-4bce-8fb9-25ca6f9270da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:22.944057 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-470c3b45-ae78-4bce-8fb9-25ca6f9270da tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097972 Aug 07 01:34:22.944741 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 827/1833] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:22 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:23.132074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-801a0915-a370-4652-970b-2b4ea07e2b90 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:23.132778 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-801a0915-a370-4652-970b-2b4ea07e2b90 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:23.139666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-801a0915-a370-4652-970b-2b4ea07e2b90 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:23.140215 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-801a0915-a370-4652-970b-2b4ea07e2b90 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:23.140767 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-801a0915-a370-4652-970b-2b4ea07e2b90 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:23.141549 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-801a0915-a370-4652-970b-2b4ea07e2b90 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:23.142058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-801a0915-a370-4652-970b-2b4ea07e2b90 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:23.142540 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-801a0915-a370-4652-970b-2b4ea07e2b90 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:23.305388 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-801a0915-a370-4652-970b-2b4ea07e2b90 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:23.315998 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-801a0915-a370-4652-970b-2b4ea07e2b90 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.187998 Aug 07 01:34:23.316697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1007/1834] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:23 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:23.975865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-60d41dac-0f17-4239-a6c0-24c941177406 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:23.976701 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-60d41dac-0f17-4239-a6c0-24c941177406 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:23.984502 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60d41dac-0f17-4239-a6c0-24c941177406 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:23.985001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60d41dac-0f17-4239-a6c0-24c941177406 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:23.985447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60d41dac-0f17-4239-a6c0-24c941177406 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:23.986174 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60d41dac-0f17-4239-a6c0-24c941177406 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:23.986668 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60d41dac-0f17-4239-a6c0-24c941177406 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:23.987101 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-60d41dac-0f17-4239-a6c0-24c941177406 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:24.057523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-60d41dac-0f17-4239-a6c0-24c941177406 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:24.066215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-60d41dac-0f17-4239-a6c0-24c941177406 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094972 Aug 07 01:34:24.066911 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 828/1835] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:23 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:24.339523 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-df46b4bb-8d6a-4169-a4e2-36db6291ef81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:24.340131 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-df46b4bb-8d6a-4169-a4e2-36db6291ef81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:24.347242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df46b4bb-8d6a-4169-a4e2-36db6291ef81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:24.347740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df46b4bb-8d6a-4169-a4e2-36db6291ef81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:24.348186 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df46b4bb-8d6a-4169-a4e2-36db6291ef81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:24.348947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df46b4bb-8d6a-4169-a4e2-36db6291ef81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:24.349406 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df46b4bb-8d6a-4169-a4e2-36db6291ef81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:24.349838 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df46b4bb-8d6a-4169-a4e2-36db6291ef81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:24.503560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-df46b4bb-8d6a-4169-a4e2-36db6291ef81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:24.512176 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-df46b4bb-8d6a-4169-a4e2-36db6291ef81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.177011 Aug 07 01:34:24.512799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1008/1836] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:24 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:25.088891 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-769085c2-0ea9-4c86-8583-3a9117a36e75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:25.089578 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-769085c2-0ea9-4c86-8583-3a9117a36e75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:25.097343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-769085c2-0ea9-4c86-8583-3a9117a36e75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:25.097886 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-769085c2-0ea9-4c86-8583-3a9117a36e75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:25.098386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-769085c2-0ea9-4c86-8583-3a9117a36e75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:25.099204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-769085c2-0ea9-4c86-8583-3a9117a36e75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:25.099716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-769085c2-0ea9-4c86-8583-3a9117a36e75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:25.100311 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-769085c2-0ea9-4c86-8583-3a9117a36e75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:25.172171 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-769085c2-0ea9-4c86-8583-3a9117a36e75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:25.180938 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-769085c2-0ea9-4c86-8583-3a9117a36e75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.096284 Aug 07 01:34:25.181541 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 829/1837] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:25 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:25.548974 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1ab5db26-581c-4cf8-b93b-abb94095232e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:25.549625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1ab5db26-581c-4cf8-b93b-abb94095232e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:25.556701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ab5db26-581c-4cf8-b93b-abb94095232e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:25.557204 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ab5db26-581c-4cf8-b93b-abb94095232e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:25.557654 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ab5db26-581c-4cf8-b93b-abb94095232e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:25.558414 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ab5db26-581c-4cf8-b93b-abb94095232e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:25.558860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ab5db26-581c-4cf8-b93b-abb94095232e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:25.559296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1ab5db26-581c-4cf8-b93b-abb94095232e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:25.714192 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1ab5db26-581c-4cf8-b93b-abb94095232e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:25.723395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1ab5db26-581c-4cf8-b93b-abb94095232e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.178473 Aug 07 01:34:25.724104 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1009/1838] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:25 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:26.203677 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1c95e840-c190-4323-bce9-86ada85c3bfb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:26.204386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1c95e840-c190-4323-bce9-86ada85c3bfb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:26.212077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1c95e840-c190-4323-bce9-86ada85c3bfb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:26.212675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1c95e840-c190-4323-bce9-86ada85c3bfb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:26.213181 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1c95e840-c190-4323-bce9-86ada85c3bfb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:26.213961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1c95e840-c190-4323-bce9-86ada85c3bfb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:26.214486 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1c95e840-c190-4323-bce9-86ada85c3bfb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:26.215137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1c95e840-c190-4323-bce9-86ada85c3bfb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:26.284280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1c95e840-c190-4323-bce9-86ada85c3bfb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:26.294201 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1c95e840-c190-4323-bce9-86ada85c3bfb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094864 Aug 07 01:34:26.294866 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 830/1839] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:26 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:26.745843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4e351e0e-1176-4a3f-93b6-9cfdf9de9382 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:26.746548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4e351e0e-1176-4a3f-93b6-9cfdf9de9382 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:26.754653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e351e0e-1176-4a3f-93b6-9cfdf9de9382 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:26.755285 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e351e0e-1176-4a3f-93b6-9cfdf9de9382 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:26.755776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e351e0e-1176-4a3f-93b6-9cfdf9de9382 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:26.756641 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e351e0e-1176-4a3f-93b6-9cfdf9de9382 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:26.757155 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e351e0e-1176-4a3f-93b6-9cfdf9de9382 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:26.757627 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4e351e0e-1176-4a3f-93b6-9cfdf9de9382 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:26.909124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4e351e0e-1176-4a3f-93b6-9cfdf9de9382 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:26.917853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4e351e0e-1176-4a3f-93b6-9cfdf9de9382 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.176068 Aug 07 01:34:26.918569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1010/1840] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:26 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:27.316179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1e86448a-af02-4996-bab9-54e6a577e468 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:27.316975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1e86448a-af02-4996-bab9-54e6a577e468 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:27.324535 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e86448a-af02-4996-bab9-54e6a577e468 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:27.325853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e86448a-af02-4996-bab9-54e6a577e468 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:27.326577 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e86448a-af02-4996-bab9-54e6a577e468 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:27.328512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e86448a-af02-4996-bab9-54e6a577e468 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:27.330268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e86448a-af02-4996-bab9-54e6a577e468 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:27.332925 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1e86448a-af02-4996-bab9-54e6a577e468 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:27.406063 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1e86448a-af02-4996-bab9-54e6a577e468 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:27.408396 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:34:27.416342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1e86448a-af02-4996-bab9-54e6a577e468 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.104186 Aug 07 01:34:27.416945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 831/1841] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:27 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:27.942509 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-991d6e61-c362-424a-9abf-b7653338d779 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:27.943230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-991d6e61-c362-424a-9abf-b7653338d779 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:27.951200 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-991d6e61-c362-424a-9abf-b7653338d779 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:27.951862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-991d6e61-c362-424a-9abf-b7653338d779 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:27.952405 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-991d6e61-c362-424a-9abf-b7653338d779 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:27.953304 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-991d6e61-c362-424a-9abf-b7653338d779 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:27.953831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-991d6e61-c362-424a-9abf-b7653338d779 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:27.954262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-991d6e61-c362-424a-9abf-b7653338d779 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:28.136294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-991d6e61-c362-424a-9abf-b7653338d779 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:28.146323 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-991d6e61-c362-424a-9abf-b7653338d779 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.208406 Aug 07 01:34:28.147027 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1011/1842] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:27 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:28.441204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ef9a2526-7466-45c1-96a7-6d32664667d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:28.442228 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ef9a2526-7466-45c1-96a7-6d32664667d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:28.452030 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef9a2526-7466-45c1-96a7-6d32664667d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:28.452765 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef9a2526-7466-45c1-96a7-6d32664667d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:28.453270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef9a2526-7466-45c1-96a7-6d32664667d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:28.454338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef9a2526-7466-45c1-96a7-6d32664667d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:28.455140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef9a2526-7466-45c1-96a7-6d32664667d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:28.455630 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef9a2526-7466-45c1-96a7-6d32664667d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:28.546197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ef9a2526-7466-45c1-96a7-6d32664667d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:28.558310 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ef9a2526-7466-45c1-96a7-6d32664667d1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.122817 Aug 07 01:34:28.558943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 832/1843] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:28 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:29.170073 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9918ba68-89e0-4fd9-9c22-582aa6f4d731 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:29.170741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9918ba68-89e0-4fd9-9c22-582aa6f4d731 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:29.177468 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9918ba68-89e0-4fd9-9c22-582aa6f4d731 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:29.177959 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9918ba68-89e0-4fd9-9c22-582aa6f4d731 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:29.178402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9918ba68-89e0-4fd9-9c22-582aa6f4d731 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:29.179126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9918ba68-89e0-4fd9-9c22-582aa6f4d731 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:29.179586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9918ba68-89e0-4fd9-9c22-582aa6f4d731 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:29.180019 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9918ba68-89e0-4fd9-9c22-582aa6f4d731 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:29.338193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9918ba68-89e0-4fd9-9c22-582aa6f4d731 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:29.347490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9918ba68-89e0-4fd9-9c22-582aa6f4d731 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.181398 Aug 07 01:34:29.348090 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1012/1844] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:29 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:29.583207 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c0a86c3f-b2cf-4b63-b93e-3e0c513e00de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:29.583859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c0a86c3f-b2cf-4b63-b93e-3e0c513e00de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:29.591417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c0a86c3f-b2cf-4b63-b93e-3e0c513e00de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:29.591917 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c0a86c3f-b2cf-4b63-b93e-3e0c513e00de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:29.592414 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c0a86c3f-b2cf-4b63-b93e-3e0c513e00de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:29.593151 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c0a86c3f-b2cf-4b63-b93e-3e0c513e00de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:29.593614 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c0a86c3f-b2cf-4b63-b93e-3e0c513e00de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:29.594049 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c0a86c3f-b2cf-4b63-b93e-3e0c513e00de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:29.662466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c0a86c3f-b2cf-4b63-b93e-3e0c513e00de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:29.671062 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c0a86c3f-b2cf-4b63-b93e-3e0c513e00de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.092003 Aug 07 01:34:29.671659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 833/1845] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:29 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:30.371320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e65f2019-d1ad-4d99-9a65-c1072ac8660e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:30.373727 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e65f2019-d1ad-4d99-9a65-c1072ac8660e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:30.382686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e65f2019-d1ad-4d99-9a65-c1072ac8660e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:30.383218 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e65f2019-d1ad-4d99-9a65-c1072ac8660e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:30.383665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e65f2019-d1ad-4d99-9a65-c1072ac8660e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:30.384631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e65f2019-d1ad-4d99-9a65-c1072ac8660e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:30.385106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e65f2019-d1ad-4d99-9a65-c1072ac8660e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:30.385546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e65f2019-d1ad-4d99-9a65-c1072ac8660e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:30.546567 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e65f2019-d1ad-4d99-9a65-c1072ac8660e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:30.560105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e65f2019-d1ad-4d99-9a65-c1072ac8660e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.192665 Aug 07 01:34:30.561184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1013/1846] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:30 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:30.704355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c29ce895-b9dc-402e-b07b-e3d7d5e261d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:30.704954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c29ce895-b9dc-402e-b07b-e3d7d5e261d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:30.712712 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c29ce895-b9dc-402e-b07b-e3d7d5e261d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:30.713373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c29ce895-b9dc-402e-b07b-e3d7d5e261d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:30.713994 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c29ce895-b9dc-402e-b07b-e3d7d5e261d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:30.714827 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c29ce895-b9dc-402e-b07b-e3d7d5e261d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:30.715957 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c29ce895-b9dc-402e-b07b-e3d7d5e261d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:30.716526 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c29ce895-b9dc-402e-b07b-e3d7d5e261d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:30.792850 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c29ce895-b9dc-402e-b07b-e3d7d5e261d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:30.802670 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c29ce895-b9dc-402e-b07b-e3d7d5e261d2 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.102248 Aug 07 01:34:30.803510 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 834/1847] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:30 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:31.583864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3d815013-fac9-4cd1-b5e8-5d63f29d3b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:31.584498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3d815013-fac9-4cd1-b5e8-5d63f29d3b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:31.587046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=52 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:34:31.593470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d815013-fac9-4cd1-b5e8-5d63f29d3b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:31.594020 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d815013-fac9-4cd1-b5e8-5d63f29d3b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:31.594508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d815013-fac9-4cd1-b5e8-5d63f29d3b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:31.595289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d815013-fac9-4cd1-b5e8-5d63f29d3b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:31.595970 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d815013-fac9-4cd1-b5e8-5d63f29d3b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:31.596811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d815013-fac9-4cd1-b5e8-5d63f29d3b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:31.751496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3d815013-fac9-4cd1-b5e8-5d63f29d3b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:31.760517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3d815013-fac9-4cd1-b5e8-5d63f29d3b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.181377 Aug 07 01:34:31.761128 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1014/1848] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:31 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:31.824876 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fb161548-f7b6-4ff9-84d8-d790c8ab01a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:31.825631 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fb161548-f7b6-4ff9-84d8-d790c8ab01a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:31.833395 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb161548-f7b6-4ff9-84d8-d790c8ab01a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:31.833978 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb161548-f7b6-4ff9-84d8-d790c8ab01a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:31.834404 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb161548-f7b6-4ff9-84d8-d790c8ab01a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:31.835169 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb161548-f7b6-4ff9-84d8-d790c8ab01a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:31.835665 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb161548-f7b6-4ff9-84d8-d790c8ab01a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:31.836122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fb161548-f7b6-4ff9-84d8-d790c8ab01a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:31.908650 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fb161548-f7b6-4ff9-84d8-d790c8ab01a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:31.922592 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fb161548-f7b6-4ff9-84d8-d790c8ab01a1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.101158 Aug 07 01:34:31.923949 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 835/1849] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:31 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:32.782315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6ae849de-6f30-4a20-898b-bd4263fbd330 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:32.782987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6ae849de-6f30-4a20-898b-bd4263fbd330 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:32.789576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ae849de-6f30-4a20-898b-bd4263fbd330 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:32.790194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ae849de-6f30-4a20-898b-bd4263fbd330 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:32.790739 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ae849de-6f30-4a20-898b-bd4263fbd330 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:32.791450 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ae849de-6f30-4a20-898b-bd4263fbd330 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:32.791899 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ae849de-6f30-4a20-898b-bd4263fbd330 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:32.792406 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ae849de-6f30-4a20-898b-bd4263fbd330 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:32.948527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-20a75b63-f59f-4c5e-8974-0e0123e53c75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:32.949754 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-20a75b63-f59f-4c5e-8974-0e0123e53c75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:32.953441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=52 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:34:32.960336 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-20a75b63-f59f-4c5e-8974-0e0123e53c75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:32.960806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-20a75b63-f59f-4c5e-8974-0e0123e53c75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:32.961245 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-20a75b63-f59f-4c5e-8974-0e0123e53c75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:32.961974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-20a75b63-f59f-4c5e-8974-0e0123e53c75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:32.962428 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-20a75b63-f59f-4c5e-8974-0e0123e53c75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:32.962861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-20a75b63-f59f-4c5e-8974-0e0123e53c75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:33.008013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6ae849de-6f30-4a20-898b-bd4263fbd330 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:33.019840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6ae849de-6f30-4a20-898b-bd4263fbd330 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.242311 Aug 07 01:34:33.021094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1015/1850] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:32 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:33.037187 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-20a75b63-f59f-4c5e-8974-0e0123e53c75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:33.046281 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-20a75b63-f59f-4c5e-8974-0e0123e53c75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.103003 Aug 07 01:34:33.046976 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 836/1851] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:32 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:34.046636 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-63151a9b-fbb1-4272-8cd6-6c6221f7f37a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:34.047311 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-63151a9b-fbb1-4272-8cd6-6c6221f7f37a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:34.055283 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63151a9b-fbb1-4272-8cd6-6c6221f7f37a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:34.055850 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63151a9b-fbb1-4272-8cd6-6c6221f7f37a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:34.056362 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63151a9b-fbb1-4272-8cd6-6c6221f7f37a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:34.057163 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63151a9b-fbb1-4272-8cd6-6c6221f7f37a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:34.057624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63151a9b-fbb1-4272-8cd6-6c6221f7f37a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:34.058080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-63151a9b-fbb1-4272-8cd6-6c6221f7f37a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:34.073710 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4c2935fd-6a51-4047-aed0-df29bf3af142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:34.074819 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4c2935fd-6a51-4047-aed0-df29bf3af142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:34.087560 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c2935fd-6a51-4047-aed0-df29bf3af142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:34.088485 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c2935fd-6a51-4047-aed0-df29bf3af142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:34.089274 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c2935fd-6a51-4047-aed0-df29bf3af142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:34.090403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c2935fd-6a51-4047-aed0-df29bf3af142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:34.091276 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c2935fd-6a51-4047-aed0-df29bf3af142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:34.092032 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c2935fd-6a51-4047-aed0-df29bf3af142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:34.198511 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4c2935fd-6a51-4047-aed0-df29bf3af142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:34.210687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4c2935fd-6a51-4047-aed0-df29bf3af142 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.144582 Aug 07 01:34:34.211473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 837/1852] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:34 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:34.228683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-63151a9b-fbb1-4272-8cd6-6c6221f7f37a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:34.244194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-63151a9b-fbb1-4272-8cd6-6c6221f7f37a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.201718 Aug 07 01:34:34.245435 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1016/1853] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:34 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:35.232603 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c7621dd1-d977-4094-a274-1d665a785b75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:35.233367 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c7621dd1-d977-4094-a274-1d665a785b75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:35.240341 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7621dd1-d977-4094-a274-1d665a785b75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:35.240895 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7621dd1-d977-4094-a274-1d665a785b75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:35.241581 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7621dd1-d977-4094-a274-1d665a785b75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:35.242419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7621dd1-d977-4094-a274-1d665a785b75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:35.242974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7621dd1-d977-4094-a274-1d665a785b75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:35.243524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c7621dd1-d977-4094-a274-1d665a785b75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:35.268805 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c6f4a0be-cab9-414b-a0d8-b95fc59d4e7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:35.269838 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c6f4a0be-cab9-414b-a0d8-b95fc59d4e7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:35.278415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c6f4a0be-cab9-414b-a0d8-b95fc59d4e7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:35.279692 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c6f4a0be-cab9-414b-a0d8-b95fc59d4e7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:35.280203 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c6f4a0be-cab9-414b-a0d8-b95fc59d4e7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:35.281189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c6f4a0be-cab9-414b-a0d8-b95fc59d4e7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:35.281952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c6f4a0be-cab9-414b-a0d8-b95fc59d4e7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:35.282513 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c6f4a0be-cab9-414b-a0d8-b95fc59d4e7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:35.322544 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c7621dd1-d977-4094-a274-1d665a785b75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:35.338984 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c7621dd1-d977-4094-a274-1d665a785b75 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.110146 Aug 07 01:34:35.340057 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 838/1854] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:35 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:35.458295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c6f4a0be-cab9-414b-a0d8-b95fc59d4e7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:35.474274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c6f4a0be-cab9-414b-a0d8-b95fc59d4e7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.209609 Aug 07 01:34:35.475348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1017/1855] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:35 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:36.369662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1565652a-93f1-4cf3-b574-538458c1965a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:36.370391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1565652a-93f1-4cf3-b574-538458c1965a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:36.377245 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1565652a-93f1-4cf3-b574-538458c1965a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:36.377744 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1565652a-93f1-4cf3-b574-538458c1965a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:36.378190 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1565652a-93f1-4cf3-b574-538458c1965a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:36.378914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1565652a-93f1-4cf3-b574-538458c1965a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:36.379385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1565652a-93f1-4cf3-b574-538458c1965a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:36.379819 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1565652a-93f1-4cf3-b574-538458c1965a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:36.455479 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1565652a-93f1-4cf3-b574-538458c1965a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:36.464889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1565652a-93f1-4cf3-b574-538458c1965a tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098965 Aug 07 01:34:36.465563 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 839/1856] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:36 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:36.501285 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-254db4f1-da70-4b11-9ded-d2dc26060c3d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:36.502193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-254db4f1-da70-4b11-9ded-d2dc26060c3d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:36.513009 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-254db4f1-da70-4b11-9ded-d2dc26060c3d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:36.514103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-254db4f1-da70-4b11-9ded-d2dc26060c3d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:36.514722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-254db4f1-da70-4b11-9ded-d2dc26060c3d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:36.516136 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-254db4f1-da70-4b11-9ded-d2dc26060c3d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:36.516960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-254db4f1-da70-4b11-9ded-d2dc26060c3d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:36.517636 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-254db4f1-da70-4b11-9ded-d2dc26060c3d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:36.675686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-254db4f1-da70-4b11-9ded-d2dc26060c3d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:36.685052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-254db4f1-da70-4b11-9ded-d2dc26060c3d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.190429 Aug 07 01:34:36.685681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1018/1857] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:36 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:37.487875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b5278e87-9ec2-41da-94c4-c77c2744a95f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:37.488626 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b5278e87-9ec2-41da-94c4-c77c2744a95f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:37.495521 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b5278e87-9ec2-41da-94c4-c77c2744a95f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:37.496024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b5278e87-9ec2-41da-94c4-c77c2744a95f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:37.497004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b5278e87-9ec2-41da-94c4-c77c2744a95f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:37.497754 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b5278e87-9ec2-41da-94c4-c77c2744a95f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:37.498255 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b5278e87-9ec2-41da-94c4-c77c2744a95f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:37.498696 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b5278e87-9ec2-41da-94c4-c77c2744a95f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:37.572332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b5278e87-9ec2-41da-94c4-c77c2744a95f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:37.581844 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b5278e87-9ec2-41da-94c4-c77c2744a95f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097493 Aug 07 01:34:37.582710 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 840/1858] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:37 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:37.707666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fa44e2ac-48ab-486e-a705-2bd9d8a6fa4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:37.708422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fa44e2ac-48ab-486e-a705-2bd9d8a6fa4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:37.716167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fa44e2ac-48ab-486e-a705-2bd9d8a6fa4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:37.716756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fa44e2ac-48ab-486e-a705-2bd9d8a6fa4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:37.717243 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fa44e2ac-48ab-486e-a705-2bd9d8a6fa4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:37.718032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fa44e2ac-48ab-486e-a705-2bd9d8a6fa4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:37.718529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fa44e2ac-48ab-486e-a705-2bd9d8a6fa4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:37.719044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fa44e2ac-48ab-486e-a705-2bd9d8a6fa4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:37.874755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-fa44e2ac-48ab-486e-a705-2bd9d8a6fa4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:37.883278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fa44e2ac-48ab-486e-a705-2bd9d8a6fa4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.179748 Aug 07 01:34:37.883880 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1019/1859] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:37 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:38.607410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a084ebd8-0be1-4a0a-9321-0279af42a916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:38.608470 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a084ebd8-0be1-4a0a-9321-0279af42a916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:38.619934 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a084ebd8-0be1-4a0a-9321-0279af42a916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:38.620830 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a084ebd8-0be1-4a0a-9321-0279af42a916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:38.622291 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a084ebd8-0be1-4a0a-9321-0279af42a916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:38.623572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a084ebd8-0be1-4a0a-9321-0279af42a916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:38.624439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a084ebd8-0be1-4a0a-9321-0279af42a916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:38.625114 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a084ebd8-0be1-4a0a-9321-0279af42a916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:38.695096 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a084ebd8-0be1-4a0a-9321-0279af42a916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:38.704545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a084ebd8-0be1-4a0a-9321-0279af42a916 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.103653 Aug 07 01:34:38.705189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 841/1860] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:38 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:38.906227 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f222c9ad-c3cd-4946-a1c4-4e39537d50fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:38.906909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f222c9ad-c3cd-4946-a1c4-4e39537d50fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:38.913829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f222c9ad-c3cd-4946-a1c4-4e39537d50fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:38.914330 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f222c9ad-c3cd-4946-a1c4-4e39537d50fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:38.914774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f222c9ad-c3cd-4946-a1c4-4e39537d50fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:38.915507 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f222c9ad-c3cd-4946-a1c4-4e39537d50fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:38.915970 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f222c9ad-c3cd-4946-a1c4-4e39537d50fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:38.916438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f222c9ad-c3cd-4946-a1c4-4e39537d50fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:38.933140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:34:39.073098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f222c9ad-c3cd-4946-a1c4-4e39537d50fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:39.085631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f222c9ad-c3cd-4946-a1c4-4e39537d50fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.183365 Aug 07 01:34:39.086715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1020/1861] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:38 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:39.726880 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ab54d340-0abb-45e1-9094-85fcb6723a71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:39.727476 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ab54d340-0abb-45e1-9094-85fcb6723a71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:39.735260 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab54d340-0abb-45e1-9094-85fcb6723a71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:39.735834 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab54d340-0abb-45e1-9094-85fcb6723a71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:39.736447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab54d340-0abb-45e1-9094-85fcb6723a71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:39.737349 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab54d340-0abb-45e1-9094-85fcb6723a71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:39.737868 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab54d340-0abb-45e1-9094-85fcb6723a71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:39.738347 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab54d340-0abb-45e1-9094-85fcb6723a71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:39.807122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ab54d340-0abb-45e1-9094-85fcb6723a71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:39.817015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ab54d340-0abb-45e1-9094-85fcb6723a71 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094150 Aug 07 01:34:39.817980 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 842/1862] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:39 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:40.108191 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c03fef9d-74a2-463b-a5fc-8bccd58793ae tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:40.109027 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c03fef9d-74a2-463b-a5fc-8bccd58793ae tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:40.116777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c03fef9d-74a2-463b-a5fc-8bccd58793ae tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:40.117360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c03fef9d-74a2-463b-a5fc-8bccd58793ae tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:40.117813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c03fef9d-74a2-463b-a5fc-8bccd58793ae tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:40.118631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c03fef9d-74a2-463b-a5fc-8bccd58793ae tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:40.119185 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c03fef9d-74a2-463b-a5fc-8bccd58793ae tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:40.119646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c03fef9d-74a2-463b-a5fc-8bccd58793ae tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:40.279432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c03fef9d-74a2-463b-a5fc-8bccd58793ae tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:40.288267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c03fef9d-74a2-463b-a5fc-8bccd58793ae tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.184557 Aug 07 01:34:40.288848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1021/1863] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:40 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:40.841852 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-deb18a25-0187-421b-b97a-6a38a917ee16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:40.842812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-deb18a25-0187-421b-b97a-6a38a917ee16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:40.850416 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-deb18a25-0187-421b-b97a-6a38a917ee16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:40.851071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-deb18a25-0187-421b-b97a-6a38a917ee16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:40.851656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-deb18a25-0187-421b-b97a-6a38a917ee16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:40.852594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-deb18a25-0187-421b-b97a-6a38a917ee16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:40.853116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-deb18a25-0187-421b-b97a-6a38a917ee16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:40.853595 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-deb18a25-0187-421b-b97a-6a38a917ee16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:40.947646 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-deb18a25-0187-421b-b97a-6a38a917ee16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:40.956923 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-deb18a25-0187-421b-b97a-6a38a917ee16 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.119882 Aug 07 01:34:40.957521 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 843/1864] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:40 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:41.311703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-15494346-4478-4417-b72b-55e818b33571 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:41.312527 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-15494346-4478-4417-b72b-55e818b33571 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:41.319282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15494346-4478-4417-b72b-55e818b33571 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:41.319777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15494346-4478-4417-b72b-55e818b33571 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:41.320219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15494346-4478-4417-b72b-55e818b33571 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:41.320969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15494346-4478-4417-b72b-55e818b33571 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:41.321429 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15494346-4478-4417-b72b-55e818b33571 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:41.321858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15494346-4478-4417-b72b-55e818b33571 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:41.512872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-15494346-4478-4417-b72b-55e818b33571 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:41.522300 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-15494346-4478-4417-b72b-55e818b33571 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.214943 Aug 07 01:34:41.523117 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1022/1865] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:41 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:41.979706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6e402414-1ea7-4f34-8e3f-57c37e0020b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:41.980459 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6e402414-1ea7-4f34-8e3f-57c37e0020b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:41.987963 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6e402414-1ea7-4f34-8e3f-57c37e0020b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:41.988489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6e402414-1ea7-4f34-8e3f-57c37e0020b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:41.988927 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6e402414-1ea7-4f34-8e3f-57c37e0020b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:41.989644 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6e402414-1ea7-4f34-8e3f-57c37e0020b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:41.990092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6e402414-1ea7-4f34-8e3f-57c37e0020b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:41.990519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6e402414-1ea7-4f34-8e3f-57c37e0020b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:42.086995 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6e402414-1ea7-4f34-8e3f-57c37e0020b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:42.098829 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6e402414-1ea7-4f34-8e3f-57c37e0020b5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.123162 Aug 07 01:34:42.099463 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 844/1866] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:41 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:42.555081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ac9aa460-0ab7-46ee-9ee0-43443877e301 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:42.555664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ac9aa460-0ab7-46ee-9ee0-43443877e301 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:42.563055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac9aa460-0ab7-46ee-9ee0-43443877e301 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:42.563550 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac9aa460-0ab7-46ee-9ee0-43443877e301 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:42.563993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac9aa460-0ab7-46ee-9ee0-43443877e301 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:42.564738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac9aa460-0ab7-46ee-9ee0-43443877e301 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:42.565194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac9aa460-0ab7-46ee-9ee0-43443877e301 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:42.565621 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ac9aa460-0ab7-46ee-9ee0-43443877e301 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:42.721465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ac9aa460-0ab7-46ee-9ee0-43443877e301 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:42.730715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ac9aa460-0ab7-46ee-9ee0-43443877e301 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.179536 Aug 07 01:34:42.731511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1023/1867] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:42 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:43.122811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-12e2f8a4-b90d-453e-8d3c-03f32dcfb8ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:43.123417 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-12e2f8a4-b90d-453e-8d3c-03f32dcfb8ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:43.130303 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-12e2f8a4-b90d-453e-8d3c-03f32dcfb8ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:43.130784 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-12e2f8a4-b90d-453e-8d3c-03f32dcfb8ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:43.131223 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-12e2f8a4-b90d-453e-8d3c-03f32dcfb8ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:43.131954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-12e2f8a4-b90d-453e-8d3c-03f32dcfb8ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:43.132449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-12e2f8a4-b90d-453e-8d3c-03f32dcfb8ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:43.132872 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-12e2f8a4-b90d-453e-8d3c-03f32dcfb8ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:43.215991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-12e2f8a4-b90d-453e-8d3c-03f32dcfb8ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:43.224930 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-12e2f8a4-b90d-453e-8d3c-03f32dcfb8ec tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.105970 Aug 07 01:34:43.225535 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 845/1868] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:43 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:43.758307 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-727b6430-034e-4f04-8cf9-c5d30b312e2e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:43.759471 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-727b6430-034e-4f04-8cf9-c5d30b312e2e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:43.771658 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-727b6430-034e-4f04-8cf9-c5d30b312e2e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:43.772585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-727b6430-034e-4f04-8cf9-c5d30b312e2e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:43.773382 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-727b6430-034e-4f04-8cf9-c5d30b312e2e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:43.774701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-727b6430-034e-4f04-8cf9-c5d30b312e2e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:43.775553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-727b6430-034e-4f04-8cf9-c5d30b312e2e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:43.776497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-727b6430-034e-4f04-8cf9-c5d30b312e2e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:43.931981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-727b6430-034e-4f04-8cf9-c5d30b312e2e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:43.940831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-727b6430-034e-4f04-8cf9-c5d30b312e2e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.190453 Aug 07 01:34:43.941438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1024/1869] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:43 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:44.247687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b808f281-fd33-445f-9595-6a6f3899bd4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:44.248768 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b808f281-fd33-445f-9595-6a6f3899bd4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:44.256025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b808f281-fd33-445f-9595-6a6f3899bd4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:44.256701 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b808f281-fd33-445f-9595-6a6f3899bd4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:44.257236 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b808f281-fd33-445f-9595-6a6f3899bd4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:44.258180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b808f281-fd33-445f-9595-6a6f3899bd4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:44.258823 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b808f281-fd33-445f-9595-6a6f3899bd4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:44.259300 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b808f281-fd33-445f-9595-6a6f3899bd4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:44.334187 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b808f281-fd33-445f-9595-6a6f3899bd4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:44.344766 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b808f281-fd33-445f-9595-6a6f3899bd4f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.101653 Aug 07 01:34:44.345280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 846/1870] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:44 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:44.975596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-81b15f9d-06a4-43ce-8f72-f1e8ba0a5e3e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:44.976474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-81b15f9d-06a4-43ce-8f72-f1e8ba0a5e3e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:44.984203 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81b15f9d-06a4-43ce-8f72-f1e8ba0a5e3e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:44.984841 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81b15f9d-06a4-43ce-8f72-f1e8ba0a5e3e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:44.985349 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81b15f9d-06a4-43ce-8f72-f1e8ba0a5e3e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:44.986112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81b15f9d-06a4-43ce-8f72-f1e8ba0a5e3e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:44.986616 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81b15f9d-06a4-43ce-8f72-f1e8ba0a5e3e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:44.987086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-81b15f9d-06a4-43ce-8f72-f1e8ba0a5e3e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:45.136691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-81b15f9d-06a4-43ce-8f72-f1e8ba0a5e3e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:45.146254 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-81b15f9d-06a4-43ce-8f72-f1e8ba0a5e3e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.174549 Aug 07 01:34:45.146860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1025/1871] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:44 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:45.366750 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6d0a000a-533a-43b7-8c19-95d11c624308 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:45.367401 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6d0a000a-533a-43b7-8c19-95d11c624308 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:45.374348 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6d0a000a-533a-43b7-8c19-95d11c624308 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:45.374842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6d0a000a-533a-43b7-8c19-95d11c624308 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:45.375286 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6d0a000a-533a-43b7-8c19-95d11c624308 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:45.376013 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6d0a000a-533a-43b7-8c19-95d11c624308 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:45.376502 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6d0a000a-533a-43b7-8c19-95d11c624308 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:45.376937 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6d0a000a-533a-43b7-8c19-95d11c624308 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:45.447153 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6d0a000a-533a-43b7-8c19-95d11c624308 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:45.458134 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6d0a000a-533a-43b7-8c19-95d11c624308 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.095267 Aug 07 01:34:45.458736 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 847/1872] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:45 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:46.168778 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3237d337-233d-41b6-a079-dcde92db3eec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:46.169372 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3237d337-233d-41b6-a079-dcde92db3eec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:46.176761 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3237d337-233d-41b6-a079-dcde92db3eec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:46.177299 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3237d337-233d-41b6-a079-dcde92db3eec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:46.177775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3237d337-233d-41b6-a079-dcde92db3eec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:46.178515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3237d337-233d-41b6-a079-dcde92db3eec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:46.178976 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3237d337-233d-41b6-a079-dcde92db3eec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:46.179407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3237d337-233d-41b6-a079-dcde92db3eec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:46.349531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3237d337-233d-41b6-a079-dcde92db3eec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:46.359724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3237d337-233d-41b6-a079-dcde92db3eec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.195529 Aug 07 01:34:46.360353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1026/1873] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:46 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:46.480740 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c96bd6f8-b8e9-46a8-b9bd-81ee5642f160 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:46.481884 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c96bd6f8-b8e9-46a8-b9bd-81ee5642f160 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:46.489502 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c96bd6f8-b8e9-46a8-b9bd-81ee5642f160 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:46.489991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c96bd6f8-b8e9-46a8-b9bd-81ee5642f160 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:46.490495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c96bd6f8-b8e9-46a8-b9bd-81ee5642f160 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:46.491232 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c96bd6f8-b8e9-46a8-b9bd-81ee5642f160 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:46.491697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c96bd6f8-b8e9-46a8-b9bd-81ee5642f160 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:46.492129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c96bd6f8-b8e9-46a8-b9bd-81ee5642f160 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:46.519695 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:34:46.597636 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c96bd6f8-b8e9-46a8-b9bd-81ee5642f160 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:46.607381 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c96bd6f8-b8e9-46a8-b9bd-81ee5642f160 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.130741 Aug 07 01:34:46.607998 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 848/1874] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:46 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:47.382639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9d59e0c5-212c-4d37-8c25-40b1bb18d3c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:47.383320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9d59e0c5-212c-4d37-8c25-40b1bb18d3c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:47.390754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d59e0c5-212c-4d37-8c25-40b1bb18d3c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:47.391342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d59e0c5-212c-4d37-8c25-40b1bb18d3c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:47.391819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d59e0c5-212c-4d37-8c25-40b1bb18d3c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:47.392635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d59e0c5-212c-4d37-8c25-40b1bb18d3c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:47.393221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d59e0c5-212c-4d37-8c25-40b1bb18d3c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:47.393694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9d59e0c5-212c-4d37-8c25-40b1bb18d3c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:47.550640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9d59e0c5-212c-4d37-8c25-40b1bb18d3c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:47.560793 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9d59e0c5-212c-4d37-8c25-40b1bb18d3c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.181901 Aug 07 01:34:47.561399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1027/1875] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:47 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:47.629697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5f8ad492-e030-4095-a6f6-3d3dccb02ec3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:47.630336 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5f8ad492-e030-4095-a6f6-3d3dccb02ec3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:47.638374 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f8ad492-e030-4095-a6f6-3d3dccb02ec3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:47.638865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f8ad492-e030-4095-a6f6-3d3dccb02ec3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:47.639307 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f8ad492-e030-4095-a6f6-3d3dccb02ec3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:47.640037 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f8ad492-e030-4095-a6f6-3d3dccb02ec3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:47.640537 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f8ad492-e030-4095-a6f6-3d3dccb02ec3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:47.640966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f8ad492-e030-4095-a6f6-3d3dccb02ec3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:47.712736 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5f8ad492-e030-4095-a6f6-3d3dccb02ec3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:47.723386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5f8ad492-e030-4095-a6f6-3d3dccb02ec3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.097635 Aug 07 01:34:47.724015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 849/1876] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:47 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:48.596929 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ee13770f-1af8-48cc-9834-fcbe975154b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:48.597617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ee13770f-1af8-48cc-9834-fcbe975154b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:48.605340 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee13770f-1af8-48cc-9834-fcbe975154b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:48.606490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee13770f-1af8-48cc-9834-fcbe975154b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:48.607208 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee13770f-1af8-48cc-9834-fcbe975154b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:48.608132 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee13770f-1af8-48cc-9834-fcbe975154b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:48.608819 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee13770f-1af8-48cc-9834-fcbe975154b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:48.609301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee13770f-1af8-48cc-9834-fcbe975154b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:48.746689 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fcd03f2f-df7b-44d8-a6f9-fe589dc58bd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:48.747790 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fcd03f2f-df7b-44d8-a6f9-fe589dc58bd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:48.758576 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fcd03f2f-df7b-44d8-a6f9-fe589dc58bd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:48.759474 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fcd03f2f-df7b-44d8-a6f9-fe589dc58bd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:48.760328 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fcd03f2f-df7b-44d8-a6f9-fe589dc58bd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:48.761681 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fcd03f2f-df7b-44d8-a6f9-fe589dc58bd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:48.763123 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fcd03f2f-df7b-44d8-a6f9-fe589dc58bd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:48.763933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fcd03f2f-df7b-44d8-a6f9-fe589dc58bd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:48.779109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ee13770f-1af8-48cc-9834-fcbe975154b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:48.789941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ee13770f-1af8-48cc-9834-fcbe975154b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.196874 Aug 07 01:34:48.790977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1028/1877] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:48 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:48.841848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fcd03f2f-df7b-44d8-a6f9-fe589dc58bd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:48.851201 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fcd03f2f-df7b-44d8-a6f9-fe589dc58bd4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.109912 Aug 07 01:34:48.851823 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 850/1878] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:48 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:49.815771 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-456a2705-eca6-4e31-bd12-0cbf5b0504c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:49.816546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-456a2705-eca6-4e31-bd12-0cbf5b0504c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:49.823534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-456a2705-eca6-4e31-bd12-0cbf5b0504c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:49.824025 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-456a2705-eca6-4e31-bd12-0cbf5b0504c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:49.824524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-456a2705-eca6-4e31-bd12-0cbf5b0504c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:49.825251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-456a2705-eca6-4e31-bd12-0cbf5b0504c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:49.825709 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-456a2705-eca6-4e31-bd12-0cbf5b0504c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:49.826139 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-456a2705-eca6-4e31-bd12-0cbf5b0504c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:49.886696 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1f4b0127-03af-422a-b2de-3f15de45ead3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:49.887616 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1f4b0127-03af-422a-b2de-3f15de45ead3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:49.895829 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f4b0127-03af-422a-b2de-3f15de45ead3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:49.896565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f4b0127-03af-422a-b2de-3f15de45ead3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:49.897245 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f4b0127-03af-422a-b2de-3f15de45ead3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:49.898182 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f4b0127-03af-422a-b2de-3f15de45ead3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:49.898878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f4b0127-03af-422a-b2de-3f15de45ead3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:49.899446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1f4b0127-03af-422a-b2de-3f15de45ead3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:49.991964 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1f4b0127-03af-422a-b2de-3f15de45ead3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:50.001185 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-456a2705-eca6-4e31-bd12-0cbf5b0504c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:50.005927 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1f4b0127-03af-422a-b2de-3f15de45ead3 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.123433 Aug 07 01:34:50.007126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 851/1879] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:49 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:50.013108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-456a2705-eca6-4e31-bd12-0cbf5b0504c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.201849 Aug 07 01:34:50.013881 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1029/1880] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:49 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:51.034253 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b43eb059-1c13-4072-915b-8ce7de78dd8f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:51.035358 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b43eb059-1c13-4072-915b-8ce7de78dd8f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:51.037370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e28d916a-7b5d-4171-9843-23f75952533c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:51.038488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e28d916a-7b5d-4171-9843-23f75952533c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:51.048430 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b43eb059-1c13-4072-915b-8ce7de78dd8f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:51.049805 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b43eb059-1c13-4072-915b-8ce7de78dd8f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:51.050042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e28d916a-7b5d-4171-9843-23f75952533c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:51.050441 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e28d916a-7b5d-4171-9843-23f75952533c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:51.050647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b43eb059-1c13-4072-915b-8ce7de78dd8f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:51.051521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e28d916a-7b5d-4171-9843-23f75952533c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:51.052429 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e28d916a-7b5d-4171-9843-23f75952533c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:51.052617 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b43eb059-1c13-4072-915b-8ce7de78dd8f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:51.052924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e28d916a-7b5d-4171-9843-23f75952533c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:51.053396 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b43eb059-1c13-4072-915b-8ce7de78dd8f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:51.053653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e28d916a-7b5d-4171-9843-23f75952533c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:51.059404 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b43eb059-1c13-4072-915b-8ce7de78dd8f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:51.166544 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b43eb059-1c13-4072-915b-8ce7de78dd8f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:51.177315 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b43eb059-1c13-4072-915b-8ce7de78dd8f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.150992 Aug 07 01:34:51.178007 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 852/1881] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:51 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:51.229728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e28d916a-7b5d-4171-9843-23f75952533c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:51.239061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e28d916a-7b5d-4171-9843-23f75952533c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.206753 Aug 07 01:34:51.239769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1030/1882] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:51 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:52.199876 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-723e5e8a-44e2-4216-a686-9fb4ba81e970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:52.200843 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-723e5e8a-44e2-4216-a686-9fb4ba81e970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:52.208226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-723e5e8a-44e2-4216-a686-9fb4ba81e970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:52.208782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-723e5e8a-44e2-4216-a686-9fb4ba81e970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:52.209249 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-723e5e8a-44e2-4216-a686-9fb4ba81e970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:52.209954 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-723e5e8a-44e2-4216-a686-9fb4ba81e970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:52.210426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-723e5e8a-44e2-4216-a686-9fb4ba81e970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:52.210860 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-723e5e8a-44e2-4216-a686-9fb4ba81e970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:52.262225 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6468ebc0-6959-47d9-8662-58b20574faff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:52.263153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6468ebc0-6959-47d9-8662-58b20574faff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:52.270412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6468ebc0-6959-47d9-8662-58b20574faff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:52.270910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6468ebc0-6959-47d9-8662-58b20574faff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:52.271357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6468ebc0-6959-47d9-8662-58b20574faff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:52.272084 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6468ebc0-6959-47d9-8662-58b20574faff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:52.272586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6468ebc0-6959-47d9-8662-58b20574faff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:52.273023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6468ebc0-6959-47d9-8662-58b20574faff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:52.283996 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-723e5e8a-44e2-4216-a686-9fb4ba81e970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:52.296640 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-723e5e8a-44e2-4216-a686-9fb4ba81e970 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.100856 Aug 07 01:34:52.297490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 853/1883] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:52 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:52.443426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6468ebc0-6959-47d9-8662-58b20574faff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:52.453396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6468ebc0-6959-47d9-8662-58b20574faff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.195049 Aug 07 01:34:52.453927 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1031/1884] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:52 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:53.331186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fbc561c1-d153-4171-adbf-75f09cd23752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:53.332100 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fbc561c1-d153-4171-adbf-75f09cd23752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:53.339140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbc561c1-d153-4171-adbf-75f09cd23752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:53.339636 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbc561c1-d153-4171-adbf-75f09cd23752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:53.340596 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbc561c1-d153-4171-adbf-75f09cd23752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:53.341333 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbc561c1-d153-4171-adbf-75f09cd23752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:53.341795 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbc561c1-d153-4171-adbf-75f09cd23752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:53.342230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbc561c1-d153-4171-adbf-75f09cd23752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:53.412082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fbc561c1-d153-4171-adbf-75f09cd23752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:53.420740 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fbc561c1-d153-4171-adbf-75f09cd23752 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.093748 Aug 07 01:34:53.421342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 854/1885] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:53 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:53.487034 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bef532c7-7e38-4fb2-99b2-ba364092d152 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:53.488119 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bef532c7-7e38-4fb2-99b2-ba364092d152 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:53.499911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bef532c7-7e38-4fb2-99b2-ba364092d152 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:53.500778 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bef532c7-7e38-4fb2-99b2-ba364092d152 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:53.501542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bef532c7-7e38-4fb2-99b2-ba364092d152 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:53.502799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bef532c7-7e38-4fb2-99b2-ba364092d152 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:53.503597 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bef532c7-7e38-4fb2-99b2-ba364092d152 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:53.504420 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bef532c7-7e38-4fb2-99b2-ba364092d152 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:53.667317 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-bef532c7-7e38-4fb2-99b2-ba364092d152 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:53.677039 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bef532c7-7e38-4fb2-99b2-ba364092d152 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.197685 Aug 07 01:34:53.677636 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1032/1886] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:53 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:54.442803 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a2454f67-3e45-44de-b178-510b6dd0f872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:54.443405 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a2454f67-3e45-44de-b178-510b6dd0f872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:54.451022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2454f67-3e45-44de-b178-510b6dd0f872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:54.451513 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2454f67-3e45-44de-b178-510b6dd0f872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:54.451955 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2454f67-3e45-44de-b178-510b6dd0f872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:54.452719 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2454f67-3e45-44de-b178-510b6dd0f872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:54.453176 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2454f67-3e45-44de-b178-510b6dd0f872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:54.453601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a2454f67-3e45-44de-b178-510b6dd0f872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:54.524709 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a2454f67-3e45-44de-b178-510b6dd0f872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:54.533998 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a2454f67-3e45-44de-b178-510b6dd0f872 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.095266 Aug 07 01:34:54.534606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 855/1887] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:54 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:54.699341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c8c5f577-f4e0-4289-a92a-af96dae9b26f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:54.700000 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c8c5f577-f4e0-4289-a92a-af96dae9b26f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:54.707229 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8c5f577-f4e0-4289-a92a-af96dae9b26f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:54.707769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8c5f577-f4e0-4289-a92a-af96dae9b26f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:54.708262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8c5f577-f4e0-4289-a92a-af96dae9b26f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:54.709072 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8c5f577-f4e0-4289-a92a-af96dae9b26f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:54.709692 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8c5f577-f4e0-4289-a92a-af96dae9b26f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:54.710168 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c8c5f577-f4e0-4289-a92a-af96dae9b26f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:54.869601 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c8c5f577-f4e0-4289-a92a-af96dae9b26f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:54.878886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c8c5f577-f4e0-4289-a92a-af96dae9b26f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.183521 Aug 07 01:34:54.879614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1033/1888] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:54 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:55.556568 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-213773ba-c1a4-4319-b308-11b7ae07d685 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:55.557206 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-213773ba-c1a4-4319-b308-11b7ae07d685 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:55.564982 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-213773ba-c1a4-4319-b308-11b7ae07d685 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:55.565483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-213773ba-c1a4-4319-b308-11b7ae07d685 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:55.565930 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-213773ba-c1a4-4319-b308-11b7ae07d685 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:55.566664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-213773ba-c1a4-4319-b308-11b7ae07d685 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:55.567129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-213773ba-c1a4-4319-b308-11b7ae07d685 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:55.567569 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-213773ba-c1a4-4319-b308-11b7ae07d685 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:55.639522 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-213773ba-c1a4-4319-b308-11b7ae07d685 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:55.648553 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-213773ba-c1a4-4319-b308-11b7ae07d685 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.096149 Aug 07 01:34:55.649156 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 856/1889] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:55 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:55.903481 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-aaa93721-c90d-4359-994d-872ebd3a1951 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:55.904214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-aaa93721-c90d-4359-994d-872ebd3a1951 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:55.911678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aaa93721-c90d-4359-994d-872ebd3a1951 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:55.912181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aaa93721-c90d-4359-994d-872ebd3a1951 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:55.912660 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aaa93721-c90d-4359-994d-872ebd3a1951 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:55.913399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aaa93721-c90d-4359-994d-872ebd3a1951 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:55.913857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aaa93721-c90d-4359-994d-872ebd3a1951 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:55.914295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aaa93721-c90d-4359-994d-872ebd3a1951 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:56.064028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-aaa93721-c90d-4359-994d-872ebd3a1951 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:56.073382 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-aaa93721-c90d-4359-994d-872ebd3a1951 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.174473 Aug 07 01:34:56.074199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1034/1890] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:55 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:56.671214 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d7fca8d3-f58a-4df5-8177-4ea1d8079fba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:56.671852 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d7fca8d3-f58a-4df5-8177-4ea1d8079fba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:56.678821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7fca8d3-f58a-4df5-8177-4ea1d8079fba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:56.679309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7fca8d3-f58a-4df5-8177-4ea1d8079fba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:56.679752 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7fca8d3-f58a-4df5-8177-4ea1d8079fba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:56.680505 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7fca8d3-f58a-4df5-8177-4ea1d8079fba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:56.680963 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7fca8d3-f58a-4df5-8177-4ea1d8079fba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:56.681386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d7fca8d3-f58a-4df5-8177-4ea1d8079fba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:56.751969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d7fca8d3-f58a-4df5-8177-4ea1d8079fba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:56.760601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d7fca8d3-f58a-4df5-8177-4ea1d8079fba tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094184 Aug 07 01:34:56.761201 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 857/1891] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:56 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:57.096916 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-171f0078-1ae5-4394-b6e6-4337b9ec60da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:57.097625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-171f0078-1ae5-4394-b6e6-4337b9ec60da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:57.104846 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-171f0078-1ae5-4394-b6e6-4337b9ec60da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:57.105339 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-171f0078-1ae5-4394-b6e6-4337b9ec60da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:57.106355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-171f0078-1ae5-4394-b6e6-4337b9ec60da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:57.107572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-171f0078-1ae5-4394-b6e6-4337b9ec60da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:57.108439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-171f0078-1ae5-4394-b6e6-4337b9ec60da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:57.109029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-171f0078-1ae5-4394-b6e6-4337b9ec60da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:57.262553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-171f0078-1ae5-4394-b6e6-4337b9ec60da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:57.271834 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-171f0078-1ae5-4394-b6e6-4337b9ec60da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.178858 Aug 07 01:34:57.272479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1035/1892] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:57 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:57.782256 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9dc99319-fc59-4d59-917e-3eadbe407902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:57.782903 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9dc99319-fc59-4d59-917e-3eadbe407902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:57.790003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9dc99319-fc59-4d59-917e-3eadbe407902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:57.790495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9dc99319-fc59-4d59-917e-3eadbe407902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:57.790944 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9dc99319-fc59-4d59-917e-3eadbe407902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:57.791679 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9dc99319-fc59-4d59-917e-3eadbe407902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:57.792144 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9dc99319-fc59-4d59-917e-3eadbe407902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:57.792607 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9dc99319-fc59-4d59-917e-3eadbe407902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:57.867186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9dc99319-fc59-4d59-917e-3eadbe407902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:57.880783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9dc99319-fc59-4d59-917e-3eadbe407902 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.102644 Aug 07 01:34:57.881937 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 858/1893] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:57 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:34:58.295799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ab493edf-50d9-4d80-b806-2cb3273e9a85 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:58.296489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ab493edf-50d9-4d80-b806-2cb3273e9a85 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:58.303906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab493edf-50d9-4d80-b806-2cb3273e9a85 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:58.304439 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab493edf-50d9-4d80-b806-2cb3273e9a85 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:58.304887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab493edf-50d9-4d80-b806-2cb3273e9a85 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:58.305635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab493edf-50d9-4d80-b806-2cb3273e9a85 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:58.306089 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab493edf-50d9-4d80-b806-2cb3273e9a85 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:58.306512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab493edf-50d9-4d80-b806-2cb3273e9a85 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:58.471083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ab493edf-50d9-4d80-b806-2cb3273e9a85 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:58.482928 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ab493edf-50d9-4d80-b806-2cb3273e9a85 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.191669 Aug 07 01:34:58.483685 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1036/1894] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:58 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:58.969203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-140a56d3-ee15-445b-bbac-7e9120454da6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:58.971219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-140a56d3-ee15-445b-bbac-7e9120454da6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:58.982408 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-140a56d3-ee15-445b-bbac-7e9120454da6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:58.982982 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-140a56d3-ee15-445b-bbac-7e9120454da6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:58.983557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-140a56d3-ee15-445b-bbac-7e9120454da6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:58.984452 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-140a56d3-ee15-445b-bbac-7e9120454da6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:58.985017 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-140a56d3-ee15-445b-bbac-7e9120454da6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:58.985547 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-140a56d3-ee15-445b-bbac-7e9120454da6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:59.059389 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-140a56d3-ee15-445b-bbac-7e9120454da6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:59.069071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-140a56d3-ee15-445b-bbac-7e9120454da6 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.105669 Aug 07 01:34:59.069840 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 859/1895] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:58 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:34:59.510271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cfe46d3e-e34f-4dce-85dd-f4656f85e1e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:34:59.511611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cfe46d3e-e34f-4dce-85dd-f4656f85e1e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:34:59.523612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cfe46d3e-e34f-4dce-85dd-f4656f85e1e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:59.524532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cfe46d3e-e34f-4dce-85dd-f4656f85e1e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:59.525303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cfe46d3e-e34f-4dce-85dd-f4656f85e1e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:59.526424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cfe46d3e-e34f-4dce-85dd-f4656f85e1e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:34:59.527094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cfe46d3e-e34f-4dce-85dd-f4656f85e1e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:34:59.527762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cfe46d3e-e34f-4dce-85dd-f4656f85e1e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:34:59.535440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:34:59.716515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-cfe46d3e-e34f-4dce-85dd-f4656f85e1e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:34:59.725127 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cfe46d3e-e34f-4dce-85dd-f4656f85e1e9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.222565 Aug 07 01:34:59.725734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1037/1896] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:34:59 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:00.092593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ccca64fd-feb7-444e-8bd1-1bf4e94b7ffb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:00.093622 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ccca64fd-feb7-444e-8bd1-1bf4e94b7ffb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:00.103477 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ccca64fd-feb7-444e-8bd1-1bf4e94b7ffb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:00.104084 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ccca64fd-feb7-444e-8bd1-1bf4e94b7ffb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:00.104793 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ccca64fd-feb7-444e-8bd1-1bf4e94b7ffb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:00.106691 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ccca64fd-feb7-444e-8bd1-1bf4e94b7ffb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:00.107294 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ccca64fd-feb7-444e-8bd1-1bf4e94b7ffb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:00.108057 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ccca64fd-feb7-444e-8bd1-1bf4e94b7ffb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:00.182588 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ccca64fd-feb7-444e-8bd1-1bf4e94b7ffb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:00.191895 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ccca64fd-feb7-444e-8bd1-1bf4e94b7ffb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.103512 Aug 07 01:35:00.192532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 860/1897] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:00 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:00.747378 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c35f94dc-b814-4160-931f-3b1a1943905f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:00.748001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c35f94dc-b814-4160-931f-3b1a1943905f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:00.755448 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c35f94dc-b814-4160-931f-3b1a1943905f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:00.755998 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c35f94dc-b814-4160-931f-3b1a1943905f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:00.756534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c35f94dc-b814-4160-931f-3b1a1943905f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:00.757376 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c35f94dc-b814-4160-931f-3b1a1943905f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:00.757886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c35f94dc-b814-4160-931f-3b1a1943905f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:00.758403 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c35f94dc-b814-4160-931f-3b1a1943905f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:00.921823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c35f94dc-b814-4160-931f-3b1a1943905f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:00.930967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c35f94dc-b814-4160-931f-3b1a1943905f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.187756 Aug 07 01:35:00.931646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1038/1898] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:00 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:01.213842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-57e8731b-1458-4675-907e-07d711a71e79 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:01.214496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-57e8731b-1458-4675-907e-07d711a71e79 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:01.222672 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57e8731b-1458-4675-907e-07d711a71e79 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:01.223321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57e8731b-1458-4675-907e-07d711a71e79 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:01.223864 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57e8731b-1458-4675-907e-07d711a71e79 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:01.224766 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57e8731b-1458-4675-907e-07d711a71e79 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:01.225339 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57e8731b-1458-4675-907e-07d711a71e79 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:01.225873 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57e8731b-1458-4675-907e-07d711a71e79 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:01.295283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-57e8731b-1458-4675-907e-07d711a71e79 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:01.304987 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-57e8731b-1458-4675-907e-07d711a71e79 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094995 Aug 07 01:35:01.305943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 861/1899] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:01 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:01.953924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a7e13282-d6d1-4502-b05a-77c3ac9479f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:01.954722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a7e13282-d6d1-4502-b05a-77c3ac9479f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:01.957296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=50 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:35:01.963082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7e13282-d6d1-4502-b05a-77c3ac9479f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:01.963574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7e13282-d6d1-4502-b05a-77c3ac9479f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:01.964018 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7e13282-d6d1-4502-b05a-77c3ac9479f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:01.964780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7e13282-d6d1-4502-b05a-77c3ac9479f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:01.965242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7e13282-d6d1-4502-b05a-77c3ac9479f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:01.965678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a7e13282-d6d1-4502-b05a-77c3ac9479f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:02.130582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a7e13282-d6d1-4502-b05a-77c3ac9479f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:02.145686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a7e13282-d6d1-4502-b05a-77c3ac9479f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.195752 Aug 07 01:35:02.146610 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1039/1900] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:01 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:02.327189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-453c9073-a1fc-499b-8491-c53f6398d73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:02.327866 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-453c9073-a1fc-499b-8491-c53f6398d73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:02.335613 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-453c9073-a1fc-499b-8491-c53f6398d73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:02.336172 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-453c9073-a1fc-499b-8491-c53f6398d73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:02.336693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-453c9073-a1fc-499b-8491-c53f6398d73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:02.337472 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-453c9073-a1fc-499b-8491-c53f6398d73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:02.337978 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-453c9073-a1fc-499b-8491-c53f6398d73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:02.338449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-453c9073-a1fc-499b-8491-c53f6398d73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:02.406778 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-453c9073-a1fc-499b-8491-c53f6398d73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:02.415707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-453c9073-a1fc-499b-8491-c53f6398d73d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.092594 Aug 07 01:35:02.416342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 862/1901] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:02 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:03.178720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-03bf7e4e-eca3-48ab-a5ae-dccf8218f828 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:03.179591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-03bf7e4e-eca3-48ab-a5ae-dccf8218f828 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:03.187050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-03bf7e4e-eca3-48ab-a5ae-dccf8218f828 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:03.187538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-03bf7e4e-eca3-48ab-a5ae-dccf8218f828 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:03.187980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-03bf7e4e-eca3-48ab-a5ae-dccf8218f828 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:03.188737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-03bf7e4e-eca3-48ab-a5ae-dccf8218f828 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:03.189196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-03bf7e4e-eca3-48ab-a5ae-dccf8218f828 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:03.189622 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-03bf7e4e-eca3-48ab-a5ae-dccf8218f828 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:03.343615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-03bf7e4e-eca3-48ab-a5ae-dccf8218f828 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:03.352797 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-03bf7e4e-eca3-48ab-a5ae-dccf8218f828 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.177823 Aug 07 01:35:03.353630 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1040/1902] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:03 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:03.438117 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f2082045-fa81-4eb5-98e6-167e98bee417 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:03.438894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f2082045-fa81-4eb5-98e6-167e98bee417 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:03.441624 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=54 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:35:03.448570 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2082045-fa81-4eb5-98e6-167e98bee417 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:03.449861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2082045-fa81-4eb5-98e6-167e98bee417 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:03.450751 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2082045-fa81-4eb5-98e6-167e98bee417 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:03.451775 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2082045-fa81-4eb5-98e6-167e98bee417 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:03.452404 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2082045-fa81-4eb5-98e6-167e98bee417 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:03.452890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f2082045-fa81-4eb5-98e6-167e98bee417 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:03.525313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f2082045-fa81-4eb5-98e6-167e98bee417 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:03.533858 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f2082045-fa81-4eb5-98e6-167e98bee417 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.099994 Aug 07 01:35:03.534466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 863/1903] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:03 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:04.379164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ace8f96b-42bf-46ca-8860-fc0e49c7c535 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:04.380260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ace8f96b-42bf-46ca-8860-fc0e49c7c535 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:04.392052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ace8f96b-42bf-46ca-8860-fc0e49c7c535 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:04.392944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ace8f96b-42bf-46ca-8860-fc0e49c7c535 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:04.393720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ace8f96b-42bf-46ca-8860-fc0e49c7c535 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:04.394975 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ace8f96b-42bf-46ca-8860-fc0e49c7c535 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:04.395809 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ace8f96b-42bf-46ca-8860-fc0e49c7c535 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:04.396629 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ace8f96b-42bf-46ca-8860-fc0e49c7c535 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:04.556799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ace8f96b-42bf-46ca-8860-fc0e49c7c535 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:04.559121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5f280ecf-08db-4661-9169-6831be157c37 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:04.560411 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5f280ecf-08db-4661-9169-6831be157c37 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:04.565536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ace8f96b-42bf-46ca-8860-fc0e49c7c535 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.194352 Aug 07 01:35:04.566140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1041/1904] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:04 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:04.571291 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f280ecf-08db-4661-9169-6831be157c37 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:04.572150 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f280ecf-08db-4661-9169-6831be157c37 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:04.572958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f280ecf-08db-4661-9169-6831be157c37 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:04.574215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f280ecf-08db-4661-9169-6831be157c37 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:04.575023 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f280ecf-08db-4661-9169-6831be157c37 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:04.575796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5f280ecf-08db-4661-9169-6831be157c37 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:04.646586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5f280ecf-08db-4661-9169-6831be157c37 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:04.656088 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5f280ecf-08db-4661-9169-6831be157c37 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.104325 Aug 07 01:35:04.656738 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 864/1905] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:04 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:05.588651 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-51c6d51d-11f9-4997-9bf8-ef5a5ac62aba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:05.589571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-51c6d51d-11f9-4997-9bf8-ef5a5ac62aba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:05.597242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51c6d51d-11f9-4997-9bf8-ef5a5ac62aba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:05.597759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51c6d51d-11f9-4997-9bf8-ef5a5ac62aba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:05.598222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51c6d51d-11f9-4997-9bf8-ef5a5ac62aba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:05.599020 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51c6d51d-11f9-4997-9bf8-ef5a5ac62aba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:05.599518 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51c6d51d-11f9-4997-9bf8-ef5a5ac62aba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:05.599961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51c6d51d-11f9-4997-9bf8-ef5a5ac62aba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:05.683434 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8260dc37-c532-42f4-8b54-0addabd553e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:05.684575 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8260dc37-c532-42f4-8b54-0addabd553e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:05.693621 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8260dc37-c532-42f4-8b54-0addabd553e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:05.694458 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8260dc37-c532-42f4-8b54-0addabd553e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:05.695191 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8260dc37-c532-42f4-8b54-0addabd553e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:05.696400 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8260dc37-c532-42f4-8b54-0addabd553e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:05.697178 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8260dc37-c532-42f4-8b54-0addabd553e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:05.697894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8260dc37-c532-42f4-8b54-0addabd553e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:05.703021 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=100 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:35:05.780440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-51c6d51d-11f9-4997-9bf8-ef5a5ac62aba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:05.789896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-51c6d51d-11f9-4997-9bf8-ef5a5ac62aba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.205928 Aug 07 01:35:05.790565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1042/1906] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:05 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:05.933189 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8260dc37-c532-42f4-8b54-0addabd553e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:05.942618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8260dc37-c532-42f4-8b54-0addabd553e0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.265137 Aug 07 01:35:05.943223 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 865/1907] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:05 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:06.812066 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-22384f5d-9942-4a46-9c37-fe1e4e7617aa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:06.812807 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-22384f5d-9942-4a46-9c37-fe1e4e7617aa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:06.819425 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-22384f5d-9942-4a46-9c37-fe1e4e7617aa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:06.819913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-22384f5d-9942-4a46-9c37-fe1e4e7617aa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:06.820409 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-22384f5d-9942-4a46-9c37-fe1e4e7617aa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:06.821117 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-22384f5d-9942-4a46-9c37-fe1e4e7617aa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:06.821571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-22384f5d-9942-4a46-9c37-fe1e4e7617aa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:06.821995 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-22384f5d-9942-4a46-9c37-fe1e4e7617aa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:06.980285 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a90fad09-dc84-45a6-a67b-98404367b712 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:06.981449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a90fad09-dc84-45a6-a67b-98404367b712 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:06.993299 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a90fad09-dc84-45a6-a67b-98404367b712 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:06.994154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a90fad09-dc84-45a6-a67b-98404367b712 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:06.994946 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a90fad09-dc84-45a6-a67b-98404367b712 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:06.996342 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a90fad09-dc84-45a6-a67b-98404367b712 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:06.997199 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a90fad09-dc84-45a6-a67b-98404367b712 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:06.999630 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a90fad09-dc84-45a6-a67b-98404367b712 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:07.066421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-22384f5d-9942-4a46-9c37-fe1e4e7617aa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:07.082989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-22384f5d-9942-4a46-9c37-fe1e4e7617aa tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.274579 Aug 07 01:35:07.084514 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1043/1908] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:06 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:07.103974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a90fad09-dc84-45a6-a67b-98404367b712 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:07.113946 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a90fad09-dc84-45a6-a67b-98404367b712 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.138112 Aug 07 01:35:07.114544 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 866/1909] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:06 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:08.119754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-94c8c777-95d2-4606-a847-e1cca4cef103 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:08.120492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-94c8c777-95d2-4606-a847-e1cca4cef103 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:08.128214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94c8c777-95d2-4606-a847-e1cca4cef103 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:08.129000 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94c8c777-95d2-4606-a847-e1cca4cef103 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:08.129765 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94c8c777-95d2-4606-a847-e1cca4cef103 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:08.130987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94c8c777-95d2-4606-a847-e1cca4cef103 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:08.131787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94c8c777-95d2-4606-a847-e1cca4cef103 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:08.132412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94c8c777-95d2-4606-a847-e1cca4cef103 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:08.136822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-80a1c798-840e-46da-be55-597240d8b8f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:08.137647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-80a1c798-840e-46da-be55-597240d8b8f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:08.147694 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80a1c798-840e-46da-be55-597240d8b8f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:08.148767 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80a1c798-840e-46da-be55-597240d8b8f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:08.149519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80a1c798-840e-46da-be55-597240d8b8f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:08.150727 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80a1c798-840e-46da-be55-597240d8b8f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:08.151553 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80a1c798-840e-46da-be55-597240d8b8f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:08.152818 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80a1c798-840e-46da-be55-597240d8b8f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:08.244527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-80a1c798-840e-46da-be55-597240d8b8f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:08.257823 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-80a1c798-840e-46da-be55-597240d8b8f4 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.124869 Aug 07 01:35:08.258423 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 867/1910] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:08 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:08.315735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-94c8c777-95d2-4606-a847-e1cca4cef103 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:08.324475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-94c8c777-95d2-4606-a847-e1cca4cef103 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.208807 Aug 07 01:35:08.325074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1044/1911] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:08 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:09.281259 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-39192b9b-64c6-4704-8646-c98723d05b8e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:09.282387 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-39192b9b-64c6-4704-8646-c98723d05b8e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:09.289219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39192b9b-64c6-4704-8646-c98723d05b8e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:09.289739 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39192b9b-64c6-4704-8646-c98723d05b8e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:09.290181 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39192b9b-64c6-4704-8646-c98723d05b8e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:09.290900 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39192b9b-64c6-4704-8646-c98723d05b8e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:09.291359 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39192b9b-64c6-4704-8646-c98723d05b8e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:09.291783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-39192b9b-64c6-4704-8646-c98723d05b8e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:09.346669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a067d0cc-b663-4cf6-86a2-43312db204d7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:09.347386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a067d0cc-b663-4cf6-86a2-43312db204d7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:09.354474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a067d0cc-b663-4cf6-86a2-43312db204d7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:09.354961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a067d0cc-b663-4cf6-86a2-43312db204d7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:09.355894 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a067d0cc-b663-4cf6-86a2-43312db204d7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:09.356656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a067d0cc-b663-4cf6-86a2-43312db204d7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:09.357114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a067d0cc-b663-4cf6-86a2-43312db204d7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:09.357539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a067d0cc-b663-4cf6-86a2-43312db204d7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:09.373615 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-39192b9b-64c6-4704-8646-c98723d05b8e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:09.383810 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-39192b9b-64c6-4704-8646-c98723d05b8e tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.106258 Aug 07 01:35:09.384630 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 868/1912] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:09 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:09.509198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a067d0cc-b663-4cf6-86a2-43312db204d7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:09.517870 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a067d0cc-b663-4cf6-86a2-43312db204d7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.175033 Aug 07 01:35:09.518456 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1045/1913] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:09 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:10.417572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cbb1632b-6b80-4962-9948-873ea824a6a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:10.418855 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-cbb1632b-6b80-4962-9948-873ea824a6a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:10.425634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbb1632b-6b80-4962-9948-873ea824a6a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:10.426129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbb1632b-6b80-4962-9948-873ea824a6a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:10.426573 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbb1632b-6b80-4962-9948-873ea824a6a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:10.427314 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbb1632b-6b80-4962-9948-873ea824a6a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:10.427772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbb1632b-6b80-4962-9948-873ea824a6a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:10.428202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbb1632b-6b80-4962-9948-873ea824a6a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:10.501451 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-cbb1632b-6b80-4962-9948-873ea824a6a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:10.516415 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-cbb1632b-6b80-4962-9948-873ea824a6a7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.102472 Aug 07 01:35:10.517849 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 869/1914] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:10 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:10.540426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b7f727fb-ee9c-4831-8912-6e7d99e6e193 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:10.541360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b7f727fb-ee9c-4831-8912-6e7d99e6e193 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:10.550951 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b7f727fb-ee9c-4831-8912-6e7d99e6e193 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:10.551538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b7f727fb-ee9c-4831-8912-6e7d99e6e193 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:10.552333 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b7f727fb-ee9c-4831-8912-6e7d99e6e193 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:10.553594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b7f727fb-ee9c-4831-8912-6e7d99e6e193 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:10.554420 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b7f727fb-ee9c-4831-8912-6e7d99e6e193 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:10.555188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b7f727fb-ee9c-4831-8912-6e7d99e6e193 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:10.732201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b7f727fb-ee9c-4831-8912-6e7d99e6e193 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:10.740781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b7f727fb-ee9c-4831-8912-6e7d99e6e193 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.205411 Aug 07 01:35:10.741390 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1046/1915] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:10 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:11.540017 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9c6322a0-e633-4604-8f4c-b89b899132e5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:11.540735 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9c6322a0-e633-4604-8f4c-b89b899132e5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:11.550564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9c6322a0-e633-4604-8f4c-b89b899132e5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:11.551461 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9c6322a0-e633-4604-8f4c-b89b899132e5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:11.552259 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9c6322a0-e633-4604-8f4c-b89b899132e5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:11.553529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9c6322a0-e633-4604-8f4c-b89b899132e5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:11.554352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9c6322a0-e633-4604-8f4c-b89b899132e5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:11.555136 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9c6322a0-e633-4604-8f4c-b89b899132e5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:11.625999 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9c6322a0-e633-4604-8f4c-b89b899132e5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:11.634529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9c6322a0-e633-4604-8f4c-b89b899132e5 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098390 Aug 07 01:35:11.635115 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 870/1916] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:11 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:11.763477 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a6ce3aa4-a5f8-4de2-9f25-fb4d547cc2f2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:11.764102 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a6ce3aa4-a5f8-4de2-9f25-fb4d547cc2f2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:11.771141 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6ce3aa4-a5f8-4de2-9f25-fb4d547cc2f2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:11.771636 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6ce3aa4-a5f8-4de2-9f25-fb4d547cc2f2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:11.772097 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6ce3aa4-a5f8-4de2-9f25-fb4d547cc2f2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:11.772837 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6ce3aa4-a5f8-4de2-9f25-fb4d547cc2f2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:11.773299 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6ce3aa4-a5f8-4de2-9f25-fb4d547cc2f2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:11.773731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6ce3aa4-a5f8-4de2-9f25-fb4d547cc2f2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:11.930976 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a6ce3aa4-a5f8-4de2-9f25-fb4d547cc2f2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:11.947985 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a6ce3aa4-a5f8-4de2-9f25-fb4d547cc2f2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.188565 Aug 07 01:35:11.950498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1047/1917] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:11 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:12.657416 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d6334cda-2545-484d-9657-bb51118e1bbc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:12.658012 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d6334cda-2545-484d-9657-bb51118e1bbc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:12.665226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d6334cda-2545-484d-9657-bb51118e1bbc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:12.665726 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d6334cda-2545-484d-9657-bb51118e1bbc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:12.666173 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d6334cda-2545-484d-9657-bb51118e1bbc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:12.666905 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d6334cda-2545-484d-9657-bb51118e1bbc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:12.667368 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d6334cda-2545-484d-9657-bb51118e1bbc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:12.667835 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d6334cda-2545-484d-9657-bb51118e1bbc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:12.751322 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d6334cda-2545-484d-9657-bb51118e1bbc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:12.759749 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d6334cda-2545-484d-9657-bb51118e1bbc tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.106389 Aug 07 01:35:12.760380 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 871/1918] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:12 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:12.974325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cdcf0523-1653-46b2-a8da-e72f67ffcf94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:12.974923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cdcf0523-1653-46b2-a8da-e72f67ffcf94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:12.982025 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cdcf0523-1653-46b2-a8da-e72f67ffcf94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:12.982523 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cdcf0523-1653-46b2-a8da-e72f67ffcf94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:12.982961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cdcf0523-1653-46b2-a8da-e72f67ffcf94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:12.983689 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cdcf0523-1653-46b2-a8da-e72f67ffcf94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:12.984153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cdcf0523-1653-46b2-a8da-e72f67ffcf94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:12.984618 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cdcf0523-1653-46b2-a8da-e72f67ffcf94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:13.135033 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-cdcf0523-1653-46b2-a8da-e72f67ffcf94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:13.144856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cdcf0523-1653-46b2-a8da-e72f67ffcf94 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.174472 Aug 07 01:35:13.145596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1048/1919] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:12 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:13.785915 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a829b18b-41a4-4fe2-85ac-cb2334b22cea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:13.787018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a829b18b-41a4-4fe2-85ac-cb2334b22cea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:13.799444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a829b18b-41a4-4fe2-85ac-cb2334b22cea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:13.800326 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a829b18b-41a4-4fe2-85ac-cb2334b22cea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:13.801091 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a829b18b-41a4-4fe2-85ac-cb2334b22cea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:13.802386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a829b18b-41a4-4fe2-85ac-cb2334b22cea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:13.803234 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a829b18b-41a4-4fe2-85ac-cb2334b22cea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:13.804010 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a829b18b-41a4-4fe2-85ac-cb2334b22cea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:13.900175 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a829b18b-41a4-4fe2-85ac-cb2334b22cea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:13.909211 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a829b18b-41a4-4fe2-85ac-cb2334b22cea tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.130592 Aug 07 01:35:13.909805 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 872/1920] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:13 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:14.167244 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-38718783-0815-4c26-a339-6d15dd507258 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:14.168037 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-38718783-0815-4c26-a339-6d15dd507258 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:14.175319 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38718783-0815-4c26-a339-6d15dd507258 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:14.175821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38718783-0815-4c26-a339-6d15dd507258 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:14.176272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38718783-0815-4c26-a339-6d15dd507258 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:14.177009 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38718783-0815-4c26-a339-6d15dd507258 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:14.177485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38718783-0815-4c26-a339-6d15dd507258 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:14.177910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-38718783-0815-4c26-a339-6d15dd507258 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:14.336916 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-38718783-0815-4c26-a339-6d15dd507258 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:14.346198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-38718783-0815-4c26-a339-6d15dd507258 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.182953 Aug 07 01:35:14.346799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1049/1921] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:14 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:14.931234 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a8b6a86f-d272-413b-bef0-5028b2c99b6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:14.931897 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a8b6a86f-d272-413b-bef0-5028b2c99b6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:14.939491 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a8b6a86f-d272-413b-bef0-5028b2c99b6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:14.939984 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a8b6a86f-d272-413b-bef0-5028b2c99b6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:14.940483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a8b6a86f-d272-413b-bef0-5028b2c99b6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:14.941215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a8b6a86f-d272-413b-bef0-5028b2c99b6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:14.941675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a8b6a86f-d272-413b-bef0-5028b2c99b6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:14.942105 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a8b6a86f-d272-413b-bef0-5028b2c99b6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:15.012653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a8b6a86f-d272-413b-bef0-5028b2c99b6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:15.021375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a8b6a86f-d272-413b-bef0-5028b2c99b6b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094074 Aug 07 01:35:15.021969 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 873/1922] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:14 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:15.367194 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7d784a43-7ea9-4c87-b685-87d1e385e46f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:15.367881 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7d784a43-7ea9-4c87-b685-87d1e385e46f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:15.375489 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d784a43-7ea9-4c87-b685-87d1e385e46f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:15.375989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d784a43-7ea9-4c87-b685-87d1e385e46f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:15.376469 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d784a43-7ea9-4c87-b685-87d1e385e46f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:15.377190 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d784a43-7ea9-4c87-b685-87d1e385e46f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:15.377649 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d784a43-7ea9-4c87-b685-87d1e385e46f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:15.378082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d784a43-7ea9-4c87-b685-87d1e385e46f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:15.531075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7d784a43-7ea9-4c87-b685-87d1e385e46f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:15.540362 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7d784a43-7ea9-4c87-b685-87d1e385e46f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.176885 Aug 07 01:35:15.540871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1050/1923] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:15 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:16.042135 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d71c4869-4bee-4301-9520-bfd147910b2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:16.043034 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d71c4869-4bee-4301-9520-bfd147910b2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:16.049635 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d71c4869-4bee-4301-9520-bfd147910b2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:16.050277 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d71c4869-4bee-4301-9520-bfd147910b2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:16.050716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d71c4869-4bee-4301-9520-bfd147910b2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:16.051455 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d71c4869-4bee-4301-9520-bfd147910b2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:16.051910 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d71c4869-4bee-4301-9520-bfd147910b2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:16.052370 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d71c4869-4bee-4301-9520-bfd147910b2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:16.124838 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d71c4869-4bee-4301-9520-bfd147910b2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:16.133281 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d71c4869-4bee-4301-9520-bfd147910b2f tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.094967 Aug 07 01:35:16.133870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 874/1924] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:16 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:16.563257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-989f02cb-640c-4326-bd15-c770805d68c3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:16.564268 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-989f02cb-640c-4326-bd15-c770805d68c3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:16.571266 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-989f02cb-640c-4326-bd15-c770805d68c3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:16.571760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-989f02cb-640c-4326-bd15-c770805d68c3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:16.572197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-989f02cb-640c-4326-bd15-c770805d68c3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:16.572940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-989f02cb-640c-4326-bd15-c770805d68c3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:16.573397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-989f02cb-640c-4326-bd15-c770805d68c3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:16.574337 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-989f02cb-640c-4326-bd15-c770805d68c3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:16.721830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-989f02cb-640c-4326-bd15-c770805d68c3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:16.730760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-989f02cb-640c-4326-bd15-c770805d68c3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.171728 Aug 07 01:35:16.731381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1051/1925] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:16 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:17.157467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cd4a9887-1141-44df-8ef8-395c2a6d42b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:17.158113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-cd4a9887-1141-44df-8ef8-395c2a6d42b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:17.165340 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd4a9887-1141-44df-8ef8-395c2a6d42b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:17.165867 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd4a9887-1141-44df-8ef8-395c2a6d42b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:17.166375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd4a9887-1141-44df-8ef8-395c2a6d42b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:17.167194 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd4a9887-1141-44df-8ef8-395c2a6d42b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:17.168468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd4a9887-1141-44df-8ef8-395c2a6d42b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:17.168981 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd4a9887-1141-44df-8ef8-395c2a6d42b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:17.253294 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-cd4a9887-1141-44df-8ef8-395c2a6d42b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:17.261778 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-cd4a9887-1141-44df-8ef8-395c2a6d42b1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.108191 Aug 07 01:35:17.262384 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 875/1926] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:17 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:17.753375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-37b6cd5c-8353-4fbd-9fbe-2b955177564d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:17.754208 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-37b6cd5c-8353-4fbd-9fbe-2b955177564d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:17.762012 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37b6cd5c-8353-4fbd-9fbe-2b955177564d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:17.762519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37b6cd5c-8353-4fbd-9fbe-2b955177564d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:17.762965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37b6cd5c-8353-4fbd-9fbe-2b955177564d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:17.763703 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37b6cd5c-8353-4fbd-9fbe-2b955177564d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:17.764221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37b6cd5c-8353-4fbd-9fbe-2b955177564d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:17.764749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37b6cd5c-8353-4fbd-9fbe-2b955177564d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:17.934328 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-37b6cd5c-8353-4fbd-9fbe-2b955177564d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:17.943436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-37b6cd5c-8353-4fbd-9fbe-2b955177564d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.194230 Aug 07 01:35:17.944037 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1052/1927] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:17 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:18.285509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ee4839ae-55c2-4419-b453-52edd7ffda9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:18.286222 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ee4839ae-55c2-4419-b453-52edd7ffda9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 6ac28640-d6da-4cfc-b7ad-0d69136ac1fd] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:18.294082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ee4839ae-55c2-4419-b453-52edd7ffda9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:18.294649 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ee4839ae-55c2-4419-b453-52edd7ffda9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:18.295232 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ee4839ae-55c2-4419-b453-52edd7ffda9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:18.296098 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ee4839ae-55c2-4419-b453-52edd7ffda9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:18.296740 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ee4839ae-55c2-4419-b453-52edd7ffda9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:18.297270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ee4839ae-55c2-4419-b453-52edd7ffda9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:18.365296 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ee4839ae-55c2-4419-b453-52edd7ffda9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:18.377973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ee4839ae-55c2-4419-b453-52edd7ffda9c tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd" status: 200 len: 1375 microversion: 2.1 time: 0.098120 Aug 07 01:35:18.378876 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 876/1928] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:18 2022] GET /compute/v2.1/servers/6ac28640-d6da-4cfc-b7ad-0d69136ac1fd => generated 1375 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:18.399115 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7922cd47-ef1d-42a1-802d-2498bc927ef7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '.version_select at 0x7f1fad5cbee0>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:18.425624 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7922cd47-ef1d-42a1-802d-2498bc927ef7 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-830225765" status: 202 len: 0 microversion: 2.1 time: 0.029686 Aug 07 01:35:18.426337 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1053/1929] 10.222.0.26 () {58 vars in 1218 bytes} [Sun Aug 7 01:35:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-830225765 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 07 01:35:18.967379 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f7ccba29-0de4-484e-82e4-d732fb13003c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:18.968003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f7ccba29-0de4-484e-82e4-d732fb13003c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:19.000558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f7ccba29-0de4-484e-82e4-d732fb13003c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:19.001548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f7ccba29-0de4-484e-82e4-d732fb13003c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:19.002603 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f7ccba29-0de4-484e-82e4-d732fb13003c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:19.003967 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f7ccba29-0de4-484e-82e4-d732fb13003c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:19.004848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f7ccba29-0de4-484e-82e4-d732fb13003c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:19.005746 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f7ccba29-0de4-484e-82e4-d732fb13003c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:19.239185 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f7ccba29-0de4-484e-82e4-d732fb13003c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:19.248169 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f7ccba29-0de4-484e-82e4-d732fb13003c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.285709 Aug 07 01:35:19.248789 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 877/1930] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:18 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:20.273407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3cda25bc-8eec-4a2f-949c-785cd3078920 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:20.274395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3cda25bc-8eec-4a2f-949c-785cd3078920 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:20.281715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3cda25bc-8eec-4a2f-949c-785cd3078920 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:20.282224 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3cda25bc-8eec-4a2f-949c-785cd3078920 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:20.282679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3cda25bc-8eec-4a2f-949c-785cd3078920 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:20.283421 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3cda25bc-8eec-4a2f-949c-785cd3078920 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:20.283923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3cda25bc-8eec-4a2f-949c-785cd3078920 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:20.284385 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3cda25bc-8eec-4a2f-949c-785cd3078920 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:20.481781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3cda25bc-8eec-4a2f-949c-785cd3078920 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:20.481781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:35:20.483687 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3cda25bc-8eec-4a2f-949c-785cd3078920 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.214422 Aug 07 01:35:20.484701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1054/1931] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:20 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:20.599069 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f9842fe4-d5dd-41ac-bfa1-1d6f0a0a3d44 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:20.599935 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f9842fe4-d5dd-41ac-bfa1-1d6f0a0a3d44 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:20.611357 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f9842fe4-d5dd-41ac-bfa1-1d6f0a0a3d44 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:20.612288 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f9842fe4-d5dd-41ac-bfa1-1d6f0a0a3d44 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:20.613043 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f9842fe4-d5dd-41ac-bfa1-1d6f0a0a3d44 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:20.624682 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f9842fe4-d5dd-41ac-bfa1-1d6f0a0a3d44 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:20.625863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-20365eae-f56f-419a-909f-13b90bd31212 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:20.626880 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-20365eae-f56f-419a-909f-13b90bd31212 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:20.632046 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f9842fe4-d5dd-41ac-bfa1-1d6f0a0a3d44 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:20.632868 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f9842fe4-d5dd-41ac-bfa1-1d6f0a0a3d44 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:20.644162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20365eae-f56f-419a-909f-13b90bd31212 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:20.644972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20365eae-f56f-419a-909f-13b90bd31212 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:20.649428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20365eae-f56f-419a-909f-13b90bd31212 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:20.653549 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20365eae-f56f-419a-909f-13b90bd31212 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:20.654510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20365eae-f56f-419a-909f-13b90bd31212 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:20.655466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-20365eae-f56f-419a-909f-13b90bd31212 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:20.663636 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-f9842fe4-d5dd-41ac-bfa1-1d6f0a0a3d44 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] HTTP exception thrown: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f9842fe4-d5dd-41ac-bfa1-1d6f0a0a3d44 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Request method failure captured: Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: DELETE /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e HTTP/1.1 Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xDuZbhbLDbrOLrZVY9vlm_HtWVxIubBKHAStAP1FnjiA_U5zjLpQJv9uy1CLGRNLdulE-dSDkxwztkKA7ofKoolemIQ0OeMpjkkOauZx4Jr-U23jKoErXsQ2oRdJqqP6y71FTJ9znyT4ExNnLWW_EBEi4UJWyaqL-JEhBPxpiLjkk0 Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Openstack-Nova-Api-Version: 2.20 Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 1b1d33b8c71e41e6bf23d4ad8590d2c2 Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:35:20.667018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:35:20.668798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:35:20.668798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:35:20.668798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 1b1d33b8c71e41e6bf23d4ad8590d2c2 Aug 07 01:35:20.668798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:35:20.668798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-AttachVolumeShelveTestJSON-734115863-project Aug 07 01:35:20.668798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:35:20.668798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:35:20.668798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: 5fafe42957914b929f9ffee53ab6e7ac Aug 07 01:35:20.668798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-AttachVolumeShelveTestJSON-734115863-project Aug 07 01:35:20.668798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:35:20.668798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:35:20.668798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '7f60fb22-d17a-4b7c-9b7d-95543be1907e'} Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:35:20.669875 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:35:20.672509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.675338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f9842fe4-d5dd-41ac-bfa1-1d6f0a0a3d44 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Returning 404 to user: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:35:20.675338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f9842fe4-d5dd-41ac-bfa1-1d6f0a0a3d44 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "DELETE /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 404 len: 111 microversion: 2.20 time: 0.078858 Aug 07 01:35:20.676973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 878/1932] 10.222.0.26 () {60 vars in 1280 bytes} [Sun Aug 7 01:35:20 2022] DELETE /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 111 bytes in 86 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 07 01:35:20.686798 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-20365eae-f56f-419a-909f-13b90bd31212 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] HTTP exception thrown: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:35:20.687906 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-78f9fbf0-d612-4c48-8c51-f34626cacad1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:20.688804 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-78f9fbf0-d612-4c48-8c51-f34626cacad1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] [instance: 7f60fb22-d17a-4b7c-9b7d-95543be1907e] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-20365eae-f56f-419a-909f-13b90bd31212 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Request method failure captured: Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: DELETE /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 HTTP/1.1 Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xD9cznP8hWf4pmxdU3hNl0ZKgrnQCZpWvAOumCUzV3uHmYQXZPbB6q2YtbZ5VRmM0fulvc4-ky89A5e1E7XmdwEvMsFQ8K0SJLQxjef3QqDDgPjeGMqBD9ErxCwISYFPrZdE0TGsrQI6E1EI8ML9K90t4_XH8zTJg5NaAH5XIw5714 Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 7fd9ab6d98a44605a57bd8ae9535a686 Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:35:20.689395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:35:20.692222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:35:20.692222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:35:20.692222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 7fd9ab6d98a44605a57bd8ae9535a686 Aug 07 01:35:20.692222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:35:20.692222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-AttachVolumeTestJSON-1659395094-project Aug 07 01:35:20.692222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:35:20.692222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:35:20.692222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 4ae9c7ef17b746fc8f87e3c22a241b76 Aug 07 01:35:20.692222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-AttachVolumeTestJSON-1659395094-project Aug 07 01:35:20.692222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:35:20.692222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:35:20.692222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '4f3f0774-3a34-4f19-9233-c4e55c7f5903'} Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:35:20.693853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:35:20.696389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.701593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-20365eae-f56f-419a-909f-13b90bd31212 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Returning 404 to user: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:35:20.701593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-20365eae-f56f-419a-909f-13b90bd31212 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "DELETE /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 404 len: 111 microversion: 2.1 time: 0.075795 Aug 07 01:35:20.701593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1055/1933] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:35:20 2022] DELETE /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 111 bytes in 78 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:35:20.702065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-78f9fbf0-d612-4c48-8c51-f34626cacad1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:20.702065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-78f9fbf0-d612-4c48-8c51-f34626cacad1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:20.702065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-78f9fbf0-d612-4c48-8c51-f34626cacad1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:20.702065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-78f9fbf0-d612-4c48-8c51-f34626cacad1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:20.702685 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-78f9fbf0-d612-4c48-8c51-f34626cacad1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:20.702685 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-78f9fbf0-d612-4c48-8c51-f34626cacad1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:20.706232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-349a9cca-d8da-4edd-acda-35beb24cbc81 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:20.709006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-349a9cca-d8da-4edd-acda-35beb24cbc81 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] [instance: 4f3f0774-3a34-4f19-9233-c4e55c7f5903] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:20.715991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-349a9cca-d8da-4edd-acda-35beb24cbc81 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:20.716765 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-349a9cca-d8da-4edd-acda-35beb24cbc81 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:20.717515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-349a9cca-d8da-4edd-acda-35beb24cbc81 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:20.718667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-349a9cca-d8da-4edd-acda-35beb24cbc81 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:20.719276 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-349a9cca-d8da-4edd-acda-35beb24cbc81 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:20.719627 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-78f9fbf0-d612-4c48-8c51-f34626cacad1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] HTTP exception thrown: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:35:20.719998 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-349a9cca-d8da-4edd-acda-35beb24cbc81 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-78f9fbf0-d612-4c48-8c51-f34626cacad1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Request method failure captured: Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e HTTP/1.1 Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xDuZbhbLDbrOLrZVY9vlm_HtWVxIubBKHAStAP1FnjiA_U5zjLpQJv9uy1CLGRNLdulE-dSDkxwztkKA7ofKoolemIQ0OeMpjkkOauZx4Jr-U23jKoErXsQ2oRdJqqP6y71FTJ9znyT4ExNnLWW_EBEi4UJWyaqL-JEhBPxpiLjkk0 Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Openstack-Nova-Api-Version: 2.20 Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 1b1d33b8c71e41e6bf23d4ad8590d2c2 Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:35:20.721263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:35:20.728266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1b1d33b8c71e41e6bf23d4ad8590d2c2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:35:20.728266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:35:20.728266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 1b1d33b8c71e41e6bf23d4ad8590d2c2 Aug 07 01:35:20.728266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-734115863 Aug 07 01:35:20.728266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-AttachVolumeShelveTestJSON-734115863-project Aug 07 01:35:20.728266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:35:20.728266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:35:20.728266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: 5fafe42957914b929f9ffee53ab6e7ac Aug 07 01:35:20.728266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-AttachVolumeShelveTestJSON-734115863-project Aug 07 01:35:20.728266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:35:20.728266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:35:20.728266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '7f60fb22-d17a-4b7c-9b7d-95543be1907e'} Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:35:20.729077 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. Aug 07 01:35:20.730394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.731537 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-78f9fbf0-d612-4c48-8c51-f34626cacad1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] Returning 404 to user: Instance 7f60fb22-d17a-4b7c-9b7d-95543be1907e could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:35:20.731537 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-78f9fbf0-d612-4c48-8c51-f34626cacad1 tempest-AttachVolumeShelveTestJSON-734115863 tempest-AttachVolumeShelveTestJSON-734115863-project] 10.222.0.26 "GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e" status: 404 len: 111 microversion: 2.20 time: 0.040831 Aug 07 01:35:20.731537 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 879/1934] 10.222.0.26 () {60 vars in 1277 bytes} [Sun Aug 7 01:35:20 2022] GET /compute/v2.1/servers/7f60fb22-d17a-4b7c-9b7d-95543be1907e => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Aug 07 01:35:20.741159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-349a9cca-d8da-4edd-acda-35beb24cbc81 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] HTTP exception thrown: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-349a9cca-d8da-4edd-acda-35beb24cbc81 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Request method failure captured: Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 HTTP/1.1 Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xD9cznP8hWf4pmxdU3hNl0ZKgrnQCZpWvAOumCUzV3uHmYQXZPbB6q2YtbZ5VRmM0fulvc4-ky89A5e1E7XmdwEvMsFQ8K0SJLQxjef3QqDDgPjeGMqBD9ErxCwISYFPrZdE0TGsrQI6E1EI8ML9K90t4_XH8zTJg5NaAH5XIw5714 Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 7fd9ab6d98a44605a57bd8ae9535a686 Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:35:20.742397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:35:20.746144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7fd9ab6d98a44605a57bd8ae9535a686"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:35:20.746144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:35:20.746144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 7fd9ab6d98a44605a57bd8ae9535a686 Aug 07 01:35:20.746144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1659395094 Aug 07 01:35:20.746144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-AttachVolumeTestJSON-1659395094-project Aug 07 01:35:20.746144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:35:20.746144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:35:20.746144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 4ae9c7ef17b746fc8f87e3c22a241b76 Aug 07 01:35:20.746144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-AttachVolumeTestJSON-1659395094-project Aug 07 01:35:20.746144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:35:20.746144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:35:20.746144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '4f3f0774-3a34-4f19-9233-c4e55c7f5903'} Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:35:20.748413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. Aug 07 01:35:20.751252 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:35:20.754373 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-349a9cca-d8da-4edd-acda-35beb24cbc81 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] Returning 404 to user: Instance 4f3f0774-3a34-4f19-9233-c4e55c7f5903 could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:35:20.754373 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-349a9cca-d8da-4edd-acda-35beb24cbc81 tempest-AttachVolumeTestJSON-1659395094 tempest-AttachVolumeTestJSON-1659395094-project] 10.222.0.26 "GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903" status: 404 len: 111 microversion: 2.1 time: 0.043460 Aug 07 01:35:20.754373 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1056/1935] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:20 2022] GET /compute/v2.1/servers/4f3f0774-3a34-4f19-9233-c4e55c7f5903 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:35:21.520753 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-caeabee3-d1e6-438c-a9c7-3cdec3d6cd0d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:21.521892 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-caeabee3-d1e6-438c-a9c7-3cdec3d6cd0d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:21.532942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-caeabee3-d1e6-438c-a9c7-3cdec3d6cd0d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:21.533854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-caeabee3-d1e6-438c-a9c7-3cdec3d6cd0d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:21.534668 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-caeabee3-d1e6-438c-a9c7-3cdec3d6cd0d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:21.535984 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-caeabee3-d1e6-438c-a9c7-3cdec3d6cd0d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:21.536885 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-caeabee3-d1e6-438c-a9c7-3cdec3d6cd0d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:21.537673 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-caeabee3-d1e6-438c-a9c7-3cdec3d6cd0d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:22.540376 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-caeabee3-d1e6-438c-a9c7-3cdec3d6cd0d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:22.550293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-caeabee3-d1e6-438c-a9c7-3cdec3d6cd0d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 1.037069 Aug 07 01:35:22.551031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 880/1936] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:21 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 1039 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:23.579292 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b033b0b4-38b4-4bee-927b-8247b34e0778 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:23.583272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b033b0b4-38b4-4bee-927b-8247b34e0778 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:23.629422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b033b0b4-38b4-4bee-927b-8247b34e0778 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:23.630367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b033b0b4-38b4-4bee-927b-8247b34e0778 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:23.630966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b033b0b4-38b4-4bee-927b-8247b34e0778 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:23.631891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b033b0b4-38b4-4bee-927b-8247b34e0778 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:23.632795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b033b0b4-38b4-4bee-927b-8247b34e0778 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:23.634087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b033b0b4-38b4-4bee-927b-8247b34e0778 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:23.896679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b033b0b4-38b4-4bee-927b-8247b34e0778 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:23.913967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b033b0b4-38b4-4bee-927b-8247b34e0778 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.342679 Aug 07 01:35:23.915247 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1057/1937] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:23 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:24.537698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:35:24.538046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:35:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:35:24.948848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-46228847-fc13-4dc2-b52c-efc8c595cb18 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:24.949692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-46228847-fc13-4dc2-b52c-efc8c595cb18 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:24.962739 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-46228847-fc13-4dc2-b52c-efc8c595cb18 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:24.963236 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-46228847-fc13-4dc2-b52c-efc8c595cb18 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:24.963680 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-46228847-fc13-4dc2-b52c-efc8c595cb18 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:24.965120 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-46228847-fc13-4dc2-b52c-efc8c595cb18 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:24.970364 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-46228847-fc13-4dc2-b52c-efc8c595cb18 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:24.971055 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-46228847-fc13-4dc2-b52c-efc8c595cb18 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:25.338588 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-46228847-fc13-4dc2-b52c-efc8c595cb18 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:25.352420 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-46228847-fc13-4dc2-b52c-efc8c595cb18 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.403931 Aug 07 01:35:25.356706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 881/1938] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:24 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:26.399758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b9d034a6-3a71-43cd-bd8a-b47d3f167c04 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:26.401405 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b9d034a6-3a71-43cd-bd8a-b47d3f167c04 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:26.425146 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9d034a6-3a71-43cd-bd8a-b47d3f167c04 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:26.426148 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9d034a6-3a71-43cd-bd8a-b47d3f167c04 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:26.427249 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9d034a6-3a71-43cd-bd8a-b47d3f167c04 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:26.429073 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9d034a6-3a71-43cd-bd8a-b47d3f167c04 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:26.431474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9d034a6-3a71-43cd-bd8a-b47d3f167c04 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:26.432477 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b9d034a6-3a71-43cd-bd8a-b47d3f167c04 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:26.833234 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b9d034a6-3a71-43cd-bd8a-b47d3f167c04 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:26.843190 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b9d034a6-3a71-43cd-bd8a-b47d3f167c04 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.451113 Aug 07 01:35:26.848585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1058/1939] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:26 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:27.885787 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7989ccde-4e90-44e4-aff0-38b29e7a23b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:27.887385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7989ccde-4e90-44e4-aff0-38b29e7a23b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:27.899585 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7989ccde-4e90-44e4-aff0-38b29e7a23b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:27.900121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7989ccde-4e90-44e4-aff0-38b29e7a23b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:27.900601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7989ccde-4e90-44e4-aff0-38b29e7a23b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:27.902203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7989ccde-4e90-44e4-aff0-38b29e7a23b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:27.902203 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7989ccde-4e90-44e4-aff0-38b29e7a23b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:27.902530 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7989ccde-4e90-44e4-aff0-38b29e7a23b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:28.202871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7989ccde-4e90-44e4-aff0-38b29e7a23b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:28.225483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7989ccde-4e90-44e4-aff0-38b29e7a23b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.343169 Aug 07 01:35:28.226496 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 882/1940] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:27 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:29.253912 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c49951e9-5910-4531-b612-e53d201d3e50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:29.254774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c49951e9-5910-4531-b612-e53d201d3e50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:29.262006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c49951e9-5910-4531-b612-e53d201d3e50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:29.262828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c49951e9-5910-4531-b612-e53d201d3e50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:29.263455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c49951e9-5910-4531-b612-e53d201d3e50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:29.264218 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c49951e9-5910-4531-b612-e53d201d3e50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:29.265046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c49951e9-5910-4531-b612-e53d201d3e50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:29.265510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c49951e9-5910-4531-b612-e53d201d3e50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:29.466598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c49951e9-5910-4531-b612-e53d201d3e50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:29.482673 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c49951e9-5910-4531-b612-e53d201d3e50 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.232348 Aug 07 01:35:29.483595 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1059/1941] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:29 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:30.229204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d8a5ac91-a36a-4da3-b16d-ebed1c746ce0 tempest-VolumesGetTestJSON-1001858442 tempest-VolumesGetTestJSON-1001858442-project] Action: 'create', calling method: .version_select at 0x7f1fad49c820>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-149368709", "metadata": {"Type": "work"}}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:35:30.519665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9f949be4-05eb-4a31-8786-a5a3f1559542 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:30.520842 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9f949be4-05eb-4a31-8786-a5a3f1559542 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:30.530792 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9f949be4-05eb-4a31-8786-a5a3f1559542 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:30.531824 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9f949be4-05eb-4a31-8786-a5a3f1559542 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:30.532550 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9f949be4-05eb-4a31-8786-a5a3f1559542 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:30.533697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9f949be4-05eb-4a31-8786-a5a3f1559542 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:30.534259 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9f949be4-05eb-4a31-8786-a5a3f1559542 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:30.534767 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9f949be4-05eb-4a31-8786-a5a3f1559542 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:30.753350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9f949be4-05eb-4a31-8786-a5a3f1559542 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:30.767055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9f949be4-05eb-4a31-8786-a5a3f1559542 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.254092 Aug 07 01:35:30.768100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1060/1942] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:30 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:31.791157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-253cc0db-6f7f-4dc0-970e-94d6bb5efd6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:31.791891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-253cc0db-6f7f-4dc0-970e-94d6bb5efd6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:31.799841 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-253cc0db-6f7f-4dc0-970e-94d6bb5efd6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:31.800563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-253cc0db-6f7f-4dc0-970e-94d6bb5efd6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:31.801060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-253cc0db-6f7f-4dc0-970e-94d6bb5efd6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:31.801813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-253cc0db-6f7f-4dc0-970e-94d6bb5efd6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:31.802339 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-253cc0db-6f7f-4dc0-970e-94d6bb5efd6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:31.802803 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-253cc0db-6f7f-4dc0-970e-94d6bb5efd6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:31.956999 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-253cc0db-6f7f-4dc0-970e-94d6bb5efd6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:31.967164 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-253cc0db-6f7f-4dc0-970e-94d6bb5efd6d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.179924 Aug 07 01:35:31.967996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1061/1943] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:31 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:32.990533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f278e0bd-0783-438e-b3a5-76608d5a26c6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:32.991313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f278e0bd-0783-438e-b3a5-76608d5a26c6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:32.993540 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=47,nova_api:DELETE=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:35:32.999380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f278e0bd-0783-438e-b3a5-76608d5a26c6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:32.999879 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f278e0bd-0783-438e-b3a5-76608d5a26c6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:33.000350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f278e0bd-0783-438e-b3a5-76608d5a26c6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:33.001076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f278e0bd-0783-438e-b3a5-76608d5a26c6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:33.001535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f278e0bd-0783-438e-b3a5-76608d5a26c6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:33.001969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f278e0bd-0783-438e-b3a5-76608d5a26c6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:33.151025 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f278e0bd-0783-438e-b3a5-76608d5a26c6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:33.161221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f278e0bd-0783-438e-b3a5-76608d5a26c6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.174646 Aug 07 01:35:33.161842 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1062/1944] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:32 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:34.184056 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-44b26c0a-ec94-4134-8c89-aa52f079b1da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:34.184835 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-44b26c0a-ec94-4134-8c89-aa52f079b1da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:34.192037 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44b26c0a-ec94-4134-8c89-aa52f079b1da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:34.192620 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44b26c0a-ec94-4134-8c89-aa52f079b1da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:34.193156 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44b26c0a-ec94-4134-8c89-aa52f079b1da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:34.193963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44b26c0a-ec94-4134-8c89-aa52f079b1da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:34.194591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44b26c0a-ec94-4134-8c89-aa52f079b1da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:34.195106 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-44b26c0a-ec94-4134-8c89-aa52f079b1da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:34.369137 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-44b26c0a-ec94-4134-8c89-aa52f079b1da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:34.384263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-44b26c0a-ec94-4134-8c89-aa52f079b1da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.204374 Aug 07 01:35:34.385339 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1063/1945] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:34 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:35.420665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-965dddd9-d3f0-4a13-b113-7086994019f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:35.421445 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-965dddd9-d3f0-4a13-b113-7086994019f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:35.428019 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-965dddd9-d3f0-4a13-b113-7086994019f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:35.428544 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-965dddd9-d3f0-4a13-b113-7086994019f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:35.429009 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-965dddd9-d3f0-4a13-b113-7086994019f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:35.429724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-965dddd9-d3f0-4a13-b113-7086994019f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:35.430177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-965dddd9-d3f0-4a13-b113-7086994019f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:35.430600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-965dddd9-d3f0-4a13-b113-7086994019f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:35.584013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-965dddd9-d3f0-4a13-b113-7086994019f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:35.593262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-965dddd9-d3f0-4a13-b113-7086994019f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.176934 Aug 07 01:35:35.593860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1064/1946] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:35 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:36.614572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9ee52fdf-07a5-4641-944f-9d840a6f8803 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:36.615309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9ee52fdf-07a5-4641-944f-9d840a6f8803 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:36.623371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9ee52fdf-07a5-4641-944f-9d840a6f8803 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:36.623862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9ee52fdf-07a5-4641-944f-9d840a6f8803 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:36.624338 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9ee52fdf-07a5-4641-944f-9d840a6f8803 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:36.625081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9ee52fdf-07a5-4641-944f-9d840a6f8803 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:36.625536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9ee52fdf-07a5-4641-944f-9d840a6f8803 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:36.625967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9ee52fdf-07a5-4641-944f-9d840a6f8803 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:36.786900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9ee52fdf-07a5-4641-944f-9d840a6f8803 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:36.796175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9ee52fdf-07a5-4641-944f-9d840a6f8803 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.185423 Aug 07 01:35:36.796799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1065/1947] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:36 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:37.818853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-aa35d24c-af2f-446b-bc4f-bab884a82a98 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:37.819591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-aa35d24c-af2f-446b-bc4f-bab884a82a98 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:37.828177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa35d24c-af2f-446b-bc4f-bab884a82a98 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:37.828811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa35d24c-af2f-446b-bc4f-bab884a82a98 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:37.829395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa35d24c-af2f-446b-bc4f-bab884a82a98 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:37.830296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa35d24c-af2f-446b-bc4f-bab884a82a98 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:37.830778 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa35d24c-af2f-446b-bc4f-bab884a82a98 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:37.831217 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa35d24c-af2f-446b-bc4f-bab884a82a98 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:37.899769 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=39 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:35:38.038841 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-aa35d24c-af2f-446b-bc4f-bab884a82a98 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:38.052545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-aa35d24c-af2f-446b-bc4f-bab884a82a98 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.237595 Aug 07 01:35:38.053634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1066/1948] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:37 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:38.221764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=99 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:35:39.079729 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c97261d3-04af-40ce-9732-22285750570f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:39.080906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c97261d3-04af-40ce-9732-22285750570f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:39.092301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c97261d3-04af-40ce-9732-22285750570f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:39.092951 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c97261d3-04af-40ce-9732-22285750570f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:39.093446 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c97261d3-04af-40ce-9732-22285750570f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:39.094237 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c97261d3-04af-40ce-9732-22285750570f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:39.094745 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c97261d3-04af-40ce-9732-22285750570f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:39.095333 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c97261d3-04af-40ce-9732-22285750570f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:39.280566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c97261d3-04af-40ce-9732-22285750570f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:39.292053 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c97261d3-04af-40ce-9732-22285750570f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.216587 Aug 07 01:35:39.292989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1067/1949] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:39 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:40.318582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-aa32edf8-f972-41bb-8f79-7510ab136f13 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:40.319750 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-aa32edf8-f972-41bb-8f79-7510ab136f13 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:40.330054 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa32edf8-f972-41bb-8f79-7510ab136f13 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:40.331162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa32edf8-f972-41bb-8f79-7510ab136f13 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:40.331461 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa32edf8-f972-41bb-8f79-7510ab136f13 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:40.332835 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa32edf8-f972-41bb-8f79-7510ab136f13 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:40.333327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa32edf8-f972-41bb-8f79-7510ab136f13 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:40.334071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aa32edf8-f972-41bb-8f79-7510ab136f13 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:40.486958 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-aa32edf8-f972-41bb-8f79-7510ab136f13 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:40.498355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-aa32edf8-f972-41bb-8f79-7510ab136f13 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.187105 Aug 07 01:35:40.499199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1068/1950] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:40 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:41.521004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-72abf123-29bd-4adf-9ebc-bb6d6695d0f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:41.521753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-72abf123-29bd-4adf-9ebc-bb6d6695d0f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:41.532183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72abf123-29bd-4adf-9ebc-bb6d6695d0f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:41.532983 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72abf123-29bd-4adf-9ebc-bb6d6695d0f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:41.533775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72abf123-29bd-4adf-9ebc-bb6d6695d0f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:41.535061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72abf123-29bd-4adf-9ebc-bb6d6695d0f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:41.535832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72abf123-29bd-4adf-9ebc-bb6d6695d0f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:41.536600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-72abf123-29bd-4adf-9ebc-bb6d6695d0f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:41.710588 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-72abf123-29bd-4adf-9ebc-bb6d6695d0f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:41.719766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-72abf123-29bd-4adf-9ebc-bb6d6695d0f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.202685 Aug 07 01:35:41.720394 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1069/1951] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:41 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:42.753459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ba67dac8-ef14-4e9d-94bf-90acc4daa2a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:42.754156 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ba67dac8-ef14-4e9d-94bf-90acc4daa2a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:42.761062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba67dac8-ef14-4e9d-94bf-90acc4daa2a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:42.761556 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba67dac8-ef14-4e9d-94bf-90acc4daa2a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:42.761995 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba67dac8-ef14-4e9d-94bf-90acc4daa2a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:42.762707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba67dac8-ef14-4e9d-94bf-90acc4daa2a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:42.763155 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba67dac8-ef14-4e9d-94bf-90acc4daa2a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:42.763578 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ba67dac8-ef14-4e9d-94bf-90acc4daa2a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:42.934366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ba67dac8-ef14-4e9d-94bf-90acc4daa2a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:42.943354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ba67dac8-ef14-4e9d-94bf-90acc4daa2a7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.194254 Aug 07 01:35:42.943957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1070/1952] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:42 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:43.967813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b0d96164-9fec-44a6-8e38-02535a308801 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:43.968942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b0d96164-9fec-44a6-8e38-02535a308801 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:43.980984 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b0d96164-9fec-44a6-8e38-02535a308801 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:43.981655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b0d96164-9fec-44a6-8e38-02535a308801 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:43.982212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b0d96164-9fec-44a6-8e38-02535a308801 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:43.983089 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b0d96164-9fec-44a6-8e38-02535a308801 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:43.983691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b0d96164-9fec-44a6-8e38-02535a308801 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:43.984272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b0d96164-9fec-44a6-8e38-02535a308801 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:44.144560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b0d96164-9fec-44a6-8e38-02535a308801 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:44.153457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b0d96164-9fec-44a6-8e38-02535a308801 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.191710 Aug 07 01:35:44.154061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1071/1953] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:43 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:45.177397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7ddb0d9b-74d0-49a8-98c9-98c92e2544e8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:45.178133 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7ddb0d9b-74d0-49a8-98c9-98c92e2544e8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:45.185643 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ddb0d9b-74d0-49a8-98c9-98c92e2544e8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:45.186332 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ddb0d9b-74d0-49a8-98c9-98c92e2544e8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:45.186823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ddb0d9b-74d0-49a8-98c9-98c92e2544e8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:45.187621 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ddb0d9b-74d0-49a8-98c9-98c92e2544e8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:45.188126 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ddb0d9b-74d0-49a8-98c9-98c92e2544e8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:45.188627 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7ddb0d9b-74d0-49a8-98c9-98c92e2544e8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:45.345959 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7ddb0d9b-74d0-49a8-98c9-98c92e2544e8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:45.349861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:35:45.361849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7ddb0d9b-74d0-49a8-98c9-98c92e2544e8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.188045 Aug 07 01:35:45.362883 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1072/1954] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:45 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:46.383608 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f5bc66e0-5025-4e78-beee-8a6c5ad8801a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:46.384775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f5bc66e0-5025-4e78-beee-8a6c5ad8801a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:46.392941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f5bc66e0-5025-4e78-beee-8a6c5ad8801a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:46.393432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f5bc66e0-5025-4e78-beee-8a6c5ad8801a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:46.393871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f5bc66e0-5025-4e78-beee-8a6c5ad8801a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:46.394586 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f5bc66e0-5025-4e78-beee-8a6c5ad8801a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:46.395059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f5bc66e0-5025-4e78-beee-8a6c5ad8801a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:46.395464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f5bc66e0-5025-4e78-beee-8a6c5ad8801a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:46.569148 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f5bc66e0-5025-4e78-beee-8a6c5ad8801a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:46.578362 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f5bc66e0-5025-4e78-beee-8a6c5ad8801a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.198940 Aug 07 01:35:46.578956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1073/1955] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:46 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:46.970032 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:35:46.970706 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:35:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-volumes (ip 10.222.0.26) !!! Aug 07 01:35:47.601224 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6a1db9a0-0d15-4f73-847b-a6e3a95ae2c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:47.602053 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6a1db9a0-0d15-4f73-847b-a6e3a95ae2c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:47.608821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6a1db9a0-0d15-4f73-847b-a6e3a95ae2c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:47.609347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6a1db9a0-0d15-4f73-847b-a6e3a95ae2c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:47.609790 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6a1db9a0-0d15-4f73-847b-a6e3a95ae2c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:47.610521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6a1db9a0-0d15-4f73-847b-a6e3a95ae2c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:47.610983 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6a1db9a0-0d15-4f73-847b-a6e3a95ae2c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:47.611415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6a1db9a0-0d15-4f73-847b-a6e3a95ae2c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:47.765881 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6a1db9a0-0d15-4f73-847b-a6e3a95ae2c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:47.775443 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6a1db9a0-0d15-4f73-847b-a6e3a95ae2c1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.178208 Aug 07 01:35:47.776054 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1074/1956] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:47 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:48.030667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:35:48.030947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:35:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:35:48.798258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e49f7c2b-ffbb-4dbb-8f05-bd1c397b64f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:48.799105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e49f7c2b-ffbb-4dbb-8f05-bd1c397b64f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:48.805873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e49f7c2b-ffbb-4dbb-8f05-bd1c397b64f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:48.806368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e49f7c2b-ffbb-4dbb-8f05-bd1c397b64f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:48.806807 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e49f7c2b-ffbb-4dbb-8f05-bd1c397b64f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:48.807575 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e49f7c2b-ffbb-4dbb-8f05-bd1c397b64f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:48.808619 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e49f7c2b-ffbb-4dbb-8f05-bd1c397b64f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:48.809055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e49f7c2b-ffbb-4dbb-8f05-bd1c397b64f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:48.958647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e49f7c2b-ffbb-4dbb-8f05-bd1c397b64f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:48.968024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e49f7c2b-ffbb-4dbb-8f05-bd1c397b64f5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.173582 Aug 07 01:35:48.968655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1075/1957] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:48 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:49.992060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3c51c3bd-5d02-4042-80e7-11021d72c5c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:49.993189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3c51c3bd-5d02-4042-80e7-11021d72c5c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:49.999101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c51c3bd-5d02-4042-80e7-11021d72c5c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:49.999592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c51c3bd-5d02-4042-80e7-11021d72c5c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:50.000035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c51c3bd-5d02-4042-80e7-11021d72c5c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:50.000794 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c51c3bd-5d02-4042-80e7-11021d72c5c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:50.001249 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c51c3bd-5d02-4042-80e7-11021d72c5c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:50.001674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c51c3bd-5d02-4042-80e7-11021d72c5c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:50.164573 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3c51c3bd-5d02-4042-80e7-11021d72c5c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:50.174805 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3c51c3bd-5d02-4042-80e7-11021d72c5c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.186819 Aug 07 01:35:50.175545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1076/1958] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:49 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:51.199405 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b5cf61a2-fb85-4623-b4d0-7d62be46723e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:51.200153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b5cf61a2-fb85-4623-b4d0-7d62be46723e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:51.206897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5cf61a2-fb85-4623-b4d0-7d62be46723e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:51.207396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5cf61a2-fb85-4623-b4d0-7d62be46723e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:51.208112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5cf61a2-fb85-4623-b4d0-7d62be46723e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:51.208861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5cf61a2-fb85-4623-b4d0-7d62be46723e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:51.209325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5cf61a2-fb85-4623-b4d0-7d62be46723e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:51.209762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b5cf61a2-fb85-4623-b4d0-7d62be46723e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:51.386351 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b5cf61a2-fb85-4623-b4d0-7d62be46723e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:51.399143 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b5cf61a2-fb85-4623-b4d0-7d62be46723e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.203485 Aug 07 01:35:51.400269 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1077/1959] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:51 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:52.422957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-13d8d317-f236-4f5c-b3de-5af18bc15b8e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:52.423693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-13d8d317-f236-4f5c-b3de-5af18bc15b8e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:52.431006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-13d8d317-f236-4f5c-b3de-5af18bc15b8e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:52.431502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-13d8d317-f236-4f5c-b3de-5af18bc15b8e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:52.431946 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-13d8d317-f236-4f5c-b3de-5af18bc15b8e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:52.432795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-13d8d317-f236-4f5c-b3de-5af18bc15b8e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:52.433282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-13d8d317-f236-4f5c-b3de-5af18bc15b8e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:52.433717 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-13d8d317-f236-4f5c-b3de-5af18bc15b8e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:52.582219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-13d8d317-f236-4f5c-b3de-5af18bc15b8e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:52.592749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-13d8d317-f236-4f5c-b3de-5af18bc15b8e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.173258 Aug 07 01:35:52.593336 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1078/1960] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:52 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:53.615766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bc79fbc7-ba51-4a45-bfa3-e3b3f87e059b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:53.616561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bc79fbc7-ba51-4a45-bfa3-e3b3f87e059b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:53.624531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc79fbc7-ba51-4a45-bfa3-e3b3f87e059b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:53.625032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc79fbc7-ba51-4a45-bfa3-e3b3f87e059b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:53.625473 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc79fbc7-ba51-4a45-bfa3-e3b3f87e059b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:53.626207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc79fbc7-ba51-4a45-bfa3-e3b3f87e059b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:53.626670 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc79fbc7-ba51-4a45-bfa3-e3b3f87e059b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:53.627105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc79fbc7-ba51-4a45-bfa3-e3b3f87e059b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:53.812584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-bc79fbc7-ba51-4a45-bfa3-e3b3f87e059b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:53.821642 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bc79fbc7-ba51-4a45-bfa3-e3b3f87e059b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.210192 Aug 07 01:35:53.822307 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1079/1961] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:53 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:54.845748 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c309e72a-2d21-4c9f-8338-d8d7c825e4a9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:54.846456 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c309e72a-2d21-4c9f-8338-d8d7c825e4a9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:54.853654 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c309e72a-2d21-4c9f-8338-d8d7c825e4a9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:54.854206 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c309e72a-2d21-4c9f-8338-d8d7c825e4a9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:54.855170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c309e72a-2d21-4c9f-8338-d8d7c825e4a9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:54.855934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c309e72a-2d21-4c9f-8338-d8d7c825e4a9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:54.856476 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c309e72a-2d21-4c9f-8338-d8d7c825e4a9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:54.856944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c309e72a-2d21-4c9f-8338-d8d7c825e4a9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:55.028698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c309e72a-2d21-4c9f-8338-d8d7c825e4a9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:55.037440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c309e72a-2d21-4c9f-8338-d8d7c825e4a9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.195618 Aug 07 01:35:55.038104 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1080/1962] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:54 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:56.059841 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-68e7f2cf-7010-4908-b11b-e24b07914397 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:56.060714 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-68e7f2cf-7010-4908-b11b-e24b07914397 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:56.067817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68e7f2cf-7010-4908-b11b-e24b07914397 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:56.068389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68e7f2cf-7010-4908-b11b-e24b07914397 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:56.068834 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68e7f2cf-7010-4908-b11b-e24b07914397 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:56.069561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68e7f2cf-7010-4908-b11b-e24b07914397 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:56.070018 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68e7f2cf-7010-4908-b11b-e24b07914397 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:56.070475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-68e7f2cf-7010-4908-b11b-e24b07914397 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:56.224816 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-68e7f2cf-7010-4908-b11b-e24b07914397 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:56.233842 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-68e7f2cf-7010-4908-b11b-e24b07914397 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.178010 Aug 07 01:35:56.234431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1081/1963] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:56 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:35:57.257815 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-340faa85-092c-45cf-8d81-558464109708 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:57.258508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-340faa85-092c-45cf-8d81-558464109708 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:57.265787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-340faa85-092c-45cf-8d81-558464109708 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:57.266280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-340faa85-092c-45cf-8d81-558464109708 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:57.266720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-340faa85-092c-45cf-8d81-558464109708 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:57.267432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-340faa85-092c-45cf-8d81-558464109708 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:57.267884 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-340faa85-092c-45cf-8d81-558464109708 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:57.268336 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-340faa85-092c-45cf-8d81-558464109708 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:57.446044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-340faa85-092c-45cf-8d81-558464109708 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:57.455455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-340faa85-092c-45cf-8d81-558464109708 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.201410 Aug 07 01:35:57.456348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1082/1964] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:57 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:58.476872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ec7c7ab5-a4f8-46a1-b96c-92e6ef6fd27f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:58.477617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ec7c7ab5-a4f8-46a1-b96c-92e6ef6fd27f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:58.484934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec7c7ab5-a4f8-46a1-b96c-92e6ef6fd27f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:58.485428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec7c7ab5-a4f8-46a1-b96c-92e6ef6fd27f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:58.485888 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec7c7ab5-a4f8-46a1-b96c-92e6ef6fd27f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:58.486615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec7c7ab5-a4f8-46a1-b96c-92e6ef6fd27f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:58.487075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec7c7ab5-a4f8-46a1-b96c-92e6ef6fd27f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:58.487507 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ec7c7ab5-a4f8-46a1-b96c-92e6ef6fd27f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:58.643433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ec7c7ab5-a4f8-46a1-b96c-92e6ef6fd27f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:58.652456 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ec7c7ab5-a4f8-46a1-b96c-92e6ef6fd27f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.179533 Aug 07 01:35:58.653050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1083/1965] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:58 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:35:59.673866 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3bd303b4-24ce-4056-a7c4-cf908fb39222 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:35:59.674908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3bd303b4-24ce-4056-a7c4-cf908fb39222 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:35:59.681592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3bd303b4-24ce-4056-a7c4-cf908fb39222 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:59.682088 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3bd303b4-24ce-4056-a7c4-cf908fb39222 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:59.683058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3bd303b4-24ce-4056-a7c4-cf908fb39222 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:59.683767 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3bd303b4-24ce-4056-a7c4-cf908fb39222 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:35:59.684261 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3bd303b4-24ce-4056-a7c4-cf908fb39222 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:35:59.684708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3bd303b4-24ce-4056-a7c4-cf908fb39222 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:35:59.889894 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3bd303b4-24ce-4056-a7c4-cf908fb39222 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:35:59.902134 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3bd303b4-24ce-4056-a7c4-cf908fb39222 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.232120 Aug 07 01:35:59.903079 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1084/1966] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:35:59 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:36:00.924011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-205682f3-689f-4663-9ca4-8a15dbf0a2cb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:00.924854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-205682f3-689f-4663-9ca4-8a15dbf0a2cb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:00.931824 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-205682f3-689f-4663-9ca4-8a15dbf0a2cb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:00.932346 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-205682f3-689f-4663-9ca4-8a15dbf0a2cb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:00.933287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-205682f3-689f-4663-9ca4-8a15dbf0a2cb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:00.934004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-205682f3-689f-4663-9ca4-8a15dbf0a2cb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:00.934467 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-205682f3-689f-4663-9ca4-8a15dbf0a2cb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:00.934895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-205682f3-689f-4663-9ca4-8a15dbf0a2cb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:01.088881 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-205682f3-689f-4663-9ca4-8a15dbf0a2cb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:01.097596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-205682f3-689f-4663-9ca4-8a15dbf0a2cb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.177605 Aug 07 01:36:01.098366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1085/1967] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:00 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:02.121748 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c63e79d4-4b75-4f53-a33a-2d04817661d5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:02.122483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c63e79d4-4b75-4f53-a33a-2d04817661d5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:02.130239 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c63e79d4-4b75-4f53-a33a-2d04817661d5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:02.130938 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c63e79d4-4b75-4f53-a33a-2d04817661d5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:02.131251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c63e79d4-4b75-4f53-a33a-2d04817661d5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:02.131961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c63e79d4-4b75-4f53-a33a-2d04817661d5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:02.132475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c63e79d4-4b75-4f53-a33a-2d04817661d5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:02.132905 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c63e79d4-4b75-4f53-a33a-2d04817661d5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:02.312398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c63e79d4-4b75-4f53-a33a-2d04817661d5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:02.321175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c63e79d4-4b75-4f53-a33a-2d04817661d5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.204345 Aug 07 01:36:02.321766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1086/1968] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:02 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:36:03.343882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-458be44b-d5e6-4ca2-9b1c-c8da897ae060 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:03.344717 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-458be44b-d5e6-4ca2-9b1c-c8da897ae060 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:03.347530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=50 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:36:03.353722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-458be44b-d5e6-4ca2-9b1c-c8da897ae060 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:03.354214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-458be44b-d5e6-4ca2-9b1c-c8da897ae060 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:03.355182 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-458be44b-d5e6-4ca2-9b1c-c8da897ae060 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:03.355912 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-458be44b-d5e6-4ca2-9b1c-c8da897ae060 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:03.356403 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-458be44b-d5e6-4ca2-9b1c-c8da897ae060 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:03.356828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-458be44b-d5e6-4ca2-9b1c-c8da897ae060 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:03.532155 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-458be44b-d5e6-4ca2-9b1c-c8da897ae060 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:03.540898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-458be44b-d5e6-4ca2-9b1c-c8da897ae060 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.200956 Aug 07 01:36:03.541498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1087/1969] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:03 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:04.563026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-37d898c3-1df5-4fdf-8784-6debf88f0f48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:04.564049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-37d898c3-1df5-4fdf-8784-6debf88f0f48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:04.570562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37d898c3-1df5-4fdf-8784-6debf88f0f48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:04.571047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37d898c3-1df5-4fdf-8784-6debf88f0f48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:04.571483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37d898c3-1df5-4fdf-8784-6debf88f0f48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:04.572181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37d898c3-1df5-4fdf-8784-6debf88f0f48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:04.572672 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37d898c3-1df5-4fdf-8784-6debf88f0f48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:04.573583 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-37d898c3-1df5-4fdf-8784-6debf88f0f48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:04.768918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-37d898c3-1df5-4fdf-8784-6debf88f0f48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:04.778490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-37d898c3-1df5-4fdf-8784-6debf88f0f48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.219369 Aug 07 01:36:04.779091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1088/1970] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:04 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:05.801372 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1b3c2d49-3737-4b53-8f51-e70de782c54e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:05.802086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1b3c2d49-3737-4b53-8f51-e70de782c54e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:05.809262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b3c2d49-3737-4b53-8f51-e70de782c54e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:05.809930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b3c2d49-3737-4b53-8f51-e70de782c54e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:05.810418 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b3c2d49-3737-4b53-8f51-e70de782c54e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:05.811184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b3c2d49-3737-4b53-8f51-e70de782c54e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:05.811718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b3c2d49-3737-4b53-8f51-e70de782c54e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:05.812195 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1b3c2d49-3737-4b53-8f51-e70de782c54e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:05.829487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:36:05.964184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1b3c2d49-3737-4b53-8f51-e70de782c54e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:05.972833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1b3c2d49-3737-4b53-8f51-e70de782c54e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.175512 Aug 07 01:36:05.973511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1089/1971] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:05 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:06.998851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8fc97675-8560-4fb7-9c69-8f7c7120439c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:06.999991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8fc97675-8560-4fb7-9c69-8f7c7120439c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:07.010573 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8fc97675-8560-4fb7-9c69-8f7c7120439c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:07.011240 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8fc97675-8560-4fb7-9c69-8f7c7120439c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:07.011787 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8fc97675-8560-4fb7-9c69-8f7c7120439c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:07.013440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8fc97675-8560-4fb7-9c69-8f7c7120439c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:07.013950 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8fc97675-8560-4fb7-9c69-8f7c7120439c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:07.014397 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8fc97675-8560-4fb7-9c69-8f7c7120439c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:07.160768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8fc97675-8560-4fb7-9c69-8f7c7120439c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:07.170091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8fc97675-8560-4fb7-9c69-8f7c7120439c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.178949 Aug 07 01:36:07.170697 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1090/1972] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:06 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:36:08.191829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-51136268-86d0-41a1-980b-17bc83cc3f81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:08.192871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-51136268-86d0-41a1-980b-17bc83cc3f81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:08.200174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51136268-86d0-41a1-980b-17bc83cc3f81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:08.200764 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51136268-86d0-41a1-980b-17bc83cc3f81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:08.201260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51136268-86d0-41a1-980b-17bc83cc3f81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:08.202188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51136268-86d0-41a1-980b-17bc83cc3f81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:08.202767 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51136268-86d0-41a1-980b-17bc83cc3f81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:08.203255 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51136268-86d0-41a1-980b-17bc83cc3f81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:08.365934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-51136268-86d0-41a1-980b-17bc83cc3f81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:08.374904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-51136268-86d0-41a1-980b-17bc83cc3f81 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.186950 Aug 07 01:36:08.375499 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1091/1973] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:08 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:09.410310 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-21aaf93b-a272-46a1-9543-270f1ee7fcba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:09.411046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-21aaf93b-a272-46a1-9543-270f1ee7fcba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:09.418117 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-21aaf93b-a272-46a1-9543-270f1ee7fcba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:09.418921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-21aaf93b-a272-46a1-9543-270f1ee7fcba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:09.419363 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-21aaf93b-a272-46a1-9543-270f1ee7fcba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:09.420094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-21aaf93b-a272-46a1-9543-270f1ee7fcba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:09.420592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-21aaf93b-a272-46a1-9543-270f1ee7fcba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:09.421027 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-21aaf93b-a272-46a1-9543-270f1ee7fcba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:09.606952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-21aaf93b-a272-46a1-9543-270f1ee7fcba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:09.615799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-21aaf93b-a272-46a1-9543-270f1ee7fcba tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.209492 Aug 07 01:36:09.616445 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1092/1974] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:09 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:10.638534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-91009579-476d-4b62-bc0f-d8bc78db2705 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:10.639631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-91009579-476d-4b62-bc0f-d8bc78db2705 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:10.646594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91009579-476d-4b62-bc0f-d8bc78db2705 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:10.647143 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91009579-476d-4b62-bc0f-d8bc78db2705 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:10.647641 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91009579-476d-4b62-bc0f-d8bc78db2705 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:10.648591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91009579-476d-4b62-bc0f-d8bc78db2705 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:10.649115 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91009579-476d-4b62-bc0f-d8bc78db2705 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:10.649632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-91009579-476d-4b62-bc0f-d8bc78db2705 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:10.801546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-91009579-476d-4b62-bc0f-d8bc78db2705 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:10.810830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-91009579-476d-4b62-bc0f-d8bc78db2705 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.176374 Aug 07 01:36:10.811508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1093/1975] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:10 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:11.833636 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8706fd0e-cc73-4884-87ba-c5757b42a00f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:11.834394 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8706fd0e-cc73-4884-87ba-c5757b42a00f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:11.842103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8706fd0e-cc73-4884-87ba-c5757b42a00f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:11.842606 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8706fd0e-cc73-4884-87ba-c5757b42a00f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:11.843057 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8706fd0e-cc73-4884-87ba-c5757b42a00f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:11.843893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8706fd0e-cc73-4884-87ba-c5757b42a00f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:11.844322 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8706fd0e-cc73-4884-87ba-c5757b42a00f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:11.844837 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8706fd0e-cc73-4884-87ba-c5757b42a00f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:12.035230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8706fd0e-cc73-4884-87ba-c5757b42a00f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:12.044415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8706fd0e-cc73-4884-87ba-c5757b42a00f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.215343 Aug 07 01:36:12.045021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1094/1976] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:11 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:13.066734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3d3f877c-599c-463f-bb60-f5a2adeb013c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:13.067517 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3d3f877c-599c-463f-bb60-f5a2adeb013c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:13.074963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d3f877c-599c-463f-bb60-f5a2adeb013c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:13.075459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d3f877c-599c-463f-bb60-f5a2adeb013c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:13.075899 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d3f877c-599c-463f-bb60-f5a2adeb013c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:13.076693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d3f877c-599c-463f-bb60-f5a2adeb013c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:13.077152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d3f877c-599c-463f-bb60-f5a2adeb013c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:13.077584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3d3f877c-599c-463f-bb60-f5a2adeb013c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:13.226860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3d3f877c-599c-463f-bb60-f5a2adeb013c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:13.236216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3d3f877c-599c-463f-bb60-f5a2adeb013c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.173391 Aug 07 01:36:13.236869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1095/1977] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:13 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:14.271433 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b05f6e49-5136-4b51-9e81-17f0e819298b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:14.272346 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b05f6e49-5136-4b51-9e81-17f0e819298b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:14.280893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b05f6e49-5136-4b51-9e81-17f0e819298b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:14.281435 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b05f6e49-5136-4b51-9e81-17f0e819298b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:14.282198 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b05f6e49-5136-4b51-9e81-17f0e819298b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:14.283427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b05f6e49-5136-4b51-9e81-17f0e819298b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:14.284205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b05f6e49-5136-4b51-9e81-17f0e819298b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:14.284994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b05f6e49-5136-4b51-9e81-17f0e819298b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:14.475605 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b05f6e49-5136-4b51-9e81-17f0e819298b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:14.490681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b05f6e49-5136-4b51-9e81-17f0e819298b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.222823 Aug 07 01:36:14.501428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1096/1978] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:14 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:36:15.518043 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a5aa3785-1577-475a-bc7f-85b48576c84d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:15.518968 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a5aa3785-1577-475a-bc7f-85b48576c84d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:15.526432 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5aa3785-1577-475a-bc7f-85b48576c84d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:15.527007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5aa3785-1577-475a-bc7f-85b48576c84d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:15.527522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5aa3785-1577-475a-bc7f-85b48576c84d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:15.528497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5aa3785-1577-475a-bc7f-85b48576c84d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:15.529112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5aa3785-1577-475a-bc7f-85b48576c84d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:15.529665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5aa3785-1577-475a-bc7f-85b48576c84d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:15.690379 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a5aa3785-1577-475a-bc7f-85b48576c84d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:15.699211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a5aa3785-1577-475a-bc7f-85b48576c84d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.185106 Aug 07 01:36:15.699811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1097/1979] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:15 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:16.720738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-36dba8c4-3664-4423-b583-bdf148804727 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:16.721505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-36dba8c4-3664-4423-b583-bdf148804727 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:16.728354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36dba8c4-3664-4423-b583-bdf148804727 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:16.728857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36dba8c4-3664-4423-b583-bdf148804727 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:16.729913 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36dba8c4-3664-4423-b583-bdf148804727 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:16.730643 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36dba8c4-3664-4423-b583-bdf148804727 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:16.731098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36dba8c4-3664-4423-b583-bdf148804727 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:16.731528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-36dba8c4-3664-4423-b583-bdf148804727 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:16.887449 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-36dba8c4-3664-4423-b583-bdf148804727 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:16.897076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-36dba8c4-3664-4423-b583-bdf148804727 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.180065 Aug 07 01:36:16.897698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1098/1980] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:16 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:17.920205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7d0a8d2d-7b25-4e57-b1dc-ceb593198854 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:17.920891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7d0a8d2d-7b25-4e57-b1dc-ceb593198854 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:17.927614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d0a8d2d-7b25-4e57-b1dc-ceb593198854 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:17.928107 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d0a8d2d-7b25-4e57-b1dc-ceb593198854 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:17.928576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d0a8d2d-7b25-4e57-b1dc-ceb593198854 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:17.929287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d0a8d2d-7b25-4e57-b1dc-ceb593198854 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:17.929745 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d0a8d2d-7b25-4e57-b1dc-ceb593198854 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:17.930184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d0a8d2d-7b25-4e57-b1dc-ceb593198854 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:18.091834 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7d0a8d2d-7b25-4e57-b1dc-ceb593198854 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:18.101215 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7d0a8d2d-7b25-4e57-b1dc-ceb593198854 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.185553 Aug 07 01:36:18.102007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1099/1981] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:17 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:19.124594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-27832d06-b871-41ab-8544-5f96ba911d1d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:19.125662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-27832d06-b871-41ab-8544-5f96ba911d1d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:19.132013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27832d06-b871-41ab-8544-5f96ba911d1d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:19.132733 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27832d06-b871-41ab-8544-5f96ba911d1d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:19.133234 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27832d06-b871-41ab-8544-5f96ba911d1d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:19.134017 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27832d06-b871-41ab-8544-5f96ba911d1d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:19.134534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27832d06-b871-41ab-8544-5f96ba911d1d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:19.135115 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-27832d06-b871-41ab-8544-5f96ba911d1d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:19.322109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-27832d06-b871-41ab-8544-5f96ba911d1d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:19.337268 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-27832d06-b871-41ab-8544-5f96ba911d1d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.216330 Aug 07 01:36:19.338339 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1100/1982] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:19 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:20.359423 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2d179f35-53d0-400d-bcd0-21c590d39a3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:20.360385 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2d179f35-53d0-400d-bcd0-21c590d39a3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:20.366990 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d179f35-53d0-400d-bcd0-21c590d39a3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:20.367591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d179f35-53d0-400d-bcd0-21c590d39a3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:20.368195 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d179f35-53d0-400d-bcd0-21c590d39a3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:20.368952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d179f35-53d0-400d-bcd0-21c590d39a3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:20.370212 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d179f35-53d0-400d-bcd0-21c590d39a3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:20.370783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2d179f35-53d0-400d-bcd0-21c590d39a3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:20.523642 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2d179f35-53d0-400d-bcd0-21c590d39a3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:20.538581 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2d179f35-53d0-400d-bcd0-21c590d39a3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.182930 Aug 07 01:36:20.539640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1101/1983] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:20 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:36:21.573578 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c09db6db-9e7f-42d6-a13d-51b7baa8f9a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:21.574279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c09db6db-9e7f-42d6-a13d-51b7baa8f9a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:21.581588 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c09db6db-9e7f-42d6-a13d-51b7baa8f9a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:21.582084 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c09db6db-9e7f-42d6-a13d-51b7baa8f9a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:21.582536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c09db6db-9e7f-42d6-a13d-51b7baa8f9a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:21.583258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c09db6db-9e7f-42d6-a13d-51b7baa8f9a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:21.583981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c09db6db-9e7f-42d6-a13d-51b7baa8f9a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:21.584471 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c09db6db-9e7f-42d6-a13d-51b7baa8f9a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:21.730445 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c09db6db-9e7f-42d6-a13d-51b7baa8f9a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:21.739129 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c09db6db-9e7f-42d6-a13d-51b7baa8f9a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.169551 Aug 07 01:36:21.739756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1102/1984] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:21 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:22.762362 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-11baed39-18ac-451d-a40a-e594304a4213 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:22.763119 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-11baed39-18ac-451d-a40a-e594304a4213 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:22.769756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-11baed39-18ac-451d-a40a-e594304a4213 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:22.770782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-11baed39-18ac-451d-a40a-e594304a4213 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:22.771243 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-11baed39-18ac-451d-a40a-e594304a4213 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:22.771977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-11baed39-18ac-451d-a40a-e594304a4213 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:22.772471 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-11baed39-18ac-451d-a40a-e594304a4213 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:22.772903 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-11baed39-18ac-451d-a40a-e594304a4213 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:22.918837 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-11baed39-18ac-451d-a40a-e594304a4213 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:22.927712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-11baed39-18ac-451d-a40a-e594304a4213 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.169340 Aug 07 01:36:22.928478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1103/1985] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:22 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:23.949878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ae94689f-4024-4127-bcdf-037e35b9e806 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:23.950630 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ae94689f-4024-4127-bcdf-037e35b9e806 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:23.957181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ae94689f-4024-4127-bcdf-037e35b9e806 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:23.957675 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ae94689f-4024-4127-bcdf-037e35b9e806 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:23.958112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ae94689f-4024-4127-bcdf-037e35b9e806 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:23.959079 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ae94689f-4024-4127-bcdf-037e35b9e806 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:23.959534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ae94689f-4024-4127-bcdf-037e35b9e806 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:23.959962 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ae94689f-4024-4127-bcdf-037e35b9e806 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:24.131917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ae94689f-4024-4127-bcdf-037e35b9e806 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:24.141037 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ae94689f-4024-4127-bcdf-037e35b9e806 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.195070 Aug 07 01:36:24.141790 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1104/1986] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:23 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:25.163915 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5d561934-713c-443c-abb1-02eb72b9a865 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:25.164768 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5d561934-713c-443c-abb1-02eb72b9a865 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:25.172789 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d561934-713c-443c-abb1-02eb72b9a865 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:25.173347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d561934-713c-443c-abb1-02eb72b9a865 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:25.173836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d561934-713c-443c-abb1-02eb72b9a865 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:25.174690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d561934-713c-443c-abb1-02eb72b9a865 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:25.175202 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d561934-713c-443c-abb1-02eb72b9a865 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:25.175691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d561934-713c-443c-abb1-02eb72b9a865 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:25.328915 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5d561934-713c-443c-abb1-02eb72b9a865 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:25.337803 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5d561934-713c-443c-abb1-02eb72b9a865 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.177672 Aug 07 01:36:25.339108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1105/1987] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:25 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:26.360474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3e9e68f8-2b36-4440-9af0-8aed22a94fcd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:26.361188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3e9e68f8-2b36-4440-9af0-8aed22a94fcd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:26.367789 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3e9e68f8-2b36-4440-9af0-8aed22a94fcd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:26.368316 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3e9e68f8-2b36-4440-9af0-8aed22a94fcd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:26.368762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3e9e68f8-2b36-4440-9af0-8aed22a94fcd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:26.370113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3e9e68f8-2b36-4440-9af0-8aed22a94fcd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:26.370568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3e9e68f8-2b36-4440-9af0-8aed22a94fcd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:26.370997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3e9e68f8-2b36-4440-9af0-8aed22a94fcd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:26.373931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:36:26.524311 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3e9e68f8-2b36-4440-9af0-8aed22a94fcd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:26.533539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3e9e68f8-2b36-4440-9af0-8aed22a94fcd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.176869 Aug 07 01:36:26.534136 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1106/1988] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:26 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:27.572754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3734aade-b2a9-4849-8873-e164589c37dd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:27.573807 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3734aade-b2a9-4849-8873-e164589c37dd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:27.582743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3734aade-b2a9-4849-8873-e164589c37dd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:27.583216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3734aade-b2a9-4849-8873-e164589c37dd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:27.583911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3734aade-b2a9-4849-8873-e164589c37dd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:27.584660 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3734aade-b2a9-4849-8873-e164589c37dd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:27.585121 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3734aade-b2a9-4849-8873-e164589c37dd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:27.585565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3734aade-b2a9-4849-8873-e164589c37dd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:27.744411 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3734aade-b2a9-4849-8873-e164589c37dd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:27.760117 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3734aade-b2a9-4849-8873-e164589c37dd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.191425 Aug 07 01:36:27.761078 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1107/1989] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:27 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:28.782208 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6d8a7699-14c5-4aea-8310-86090cabd3d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:28.783096 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6d8a7699-14c5-4aea-8310-86090cabd3d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:28.790744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d8a7699-14c5-4aea-8310-86090cabd3d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:28.791854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d8a7699-14c5-4aea-8310-86090cabd3d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:28.792402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d8a7699-14c5-4aea-8310-86090cabd3d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:28.793195 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d8a7699-14c5-4aea-8310-86090cabd3d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:28.793708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d8a7699-14c5-4aea-8310-86090cabd3d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:28.794274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6d8a7699-14c5-4aea-8310-86090cabd3d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:28.943294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6d8a7699-14c5-4aea-8310-86090cabd3d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:28.952514 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6d8a7699-14c5-4aea-8310-86090cabd3d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.174182 Aug 07 01:36:28.953117 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1108/1990] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:28 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:36:29.975733 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-15aea22d-91ce-4a98-953a-9041f6149ef9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:29.976527 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-15aea22d-91ce-4a98-953a-9041f6149ef9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:29.984274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15aea22d-91ce-4a98-953a-9041f6149ef9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:29.984846 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15aea22d-91ce-4a98-953a-9041f6149ef9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:29.985291 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15aea22d-91ce-4a98-953a-9041f6149ef9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:29.986071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15aea22d-91ce-4a98-953a-9041f6149ef9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:29.986526 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15aea22d-91ce-4a98-953a-9041f6149ef9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:29.987037 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15aea22d-91ce-4a98-953a-9041f6149ef9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:30.185389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-15aea22d-91ce-4a98-953a-9041f6149ef9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:30.197023 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-15aea22d-91ce-4a98-953a-9041f6149ef9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.225766 Aug 07 01:36:30.197731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1109/1991] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:29 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:31.222982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0d850b5e-6195-4ab6-b8dc-30a877b94494 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:31.223955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0d850b5e-6195-4ab6-b8dc-30a877b94494 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:31.235068 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d850b5e-6195-4ab6-b8dc-30a877b94494 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:31.235566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d850b5e-6195-4ab6-b8dc-30a877b94494 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:31.236004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d850b5e-6195-4ab6-b8dc-30a877b94494 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:31.236798 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d850b5e-6195-4ab6-b8dc-30a877b94494 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:31.237263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d850b5e-6195-4ab6-b8dc-30a877b94494 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:31.237695 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d850b5e-6195-4ab6-b8dc-30a877b94494 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:31.444853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0d850b5e-6195-4ab6-b8dc-30a877b94494 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:31.458296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0d850b5e-6195-4ab6-b8dc-30a877b94494 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.239288 Aug 07 01:36:31.459367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1110/1992] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:31 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:32.482601 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-761b906e-171e-4551-9a96-3dd83238a5b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:32.483343 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-761b906e-171e-4551-9a96-3dd83238a5b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:32.490288 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-761b906e-171e-4551-9a96-3dd83238a5b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:32.490839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-761b906e-171e-4551-9a96-3dd83238a5b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:32.491389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-761b906e-171e-4551-9a96-3dd83238a5b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:32.492372 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-761b906e-171e-4551-9a96-3dd83238a5b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:32.492907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-761b906e-171e-4551-9a96-3dd83238a5b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:32.493455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-761b906e-171e-4551-9a96-3dd83238a5b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:32.645247 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-761b906e-171e-4551-9a96-3dd83238a5b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:32.657540 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-761b906e-171e-4551-9a96-3dd83238a5b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.178564 Aug 07 01:36:32.658430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1111/1993] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:32 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:33.687448 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-88d36e05-cd92-4fd8-85c4-fecc8234524e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:33.688277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-88d36e05-cd92-4fd8-85c4-fecc8234524e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:33.690783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=50 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:36:33.697505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88d36e05-cd92-4fd8-85c4-fecc8234524e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:33.698024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88d36e05-cd92-4fd8-85c4-fecc8234524e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:33.698463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88d36e05-cd92-4fd8-85c4-fecc8234524e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:33.699199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88d36e05-cd92-4fd8-85c4-fecc8234524e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:33.699686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88d36e05-cd92-4fd8-85c4-fecc8234524e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:33.700108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-88d36e05-cd92-4fd8-85c4-fecc8234524e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:33.850107 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-88d36e05-cd92-4fd8-85c4-fecc8234524e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:33.866279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-88d36e05-cd92-4fd8-85c4-fecc8234524e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.182307 Aug 07 01:36:33.867398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1112/1994] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:33 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:34.890183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-15c5d024-9156-48da-b76d-bf9b72a63fa6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:34.891118 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-15c5d024-9156-48da-b76d-bf9b72a63fa6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:34.897770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15c5d024-9156-48da-b76d-bf9b72a63fa6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:34.898275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15c5d024-9156-48da-b76d-bf9b72a63fa6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:34.898776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15c5d024-9156-48da-b76d-bf9b72a63fa6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:34.899495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15c5d024-9156-48da-b76d-bf9b72a63fa6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:34.899958 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15c5d024-9156-48da-b76d-bf9b72a63fa6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:34.900424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15c5d024-9156-48da-b76d-bf9b72a63fa6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:35.045027 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-15c5d024-9156-48da-b76d-bf9b72a63fa6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:35.053490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-15c5d024-9156-48da-b76d-bf9b72a63fa6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.167761 Aug 07 01:36:35.054101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1113/1995] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:34 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:36.090052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e17f72c8-b5f2-4a66-8bfa-a27a42a59294 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:36.090801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e17f72c8-b5f2-4a66-8bfa-a27a42a59294 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:36.098607 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e17f72c8-b5f2-4a66-8bfa-a27a42a59294 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:36.098607 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e17f72c8-b5f2-4a66-8bfa-a27a42a59294 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:36.098607 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e17f72c8-b5f2-4a66-8bfa-a27a42a59294 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:36.099151 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e17f72c8-b5f2-4a66-8bfa-a27a42a59294 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:36.099516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e17f72c8-b5f2-4a66-8bfa-a27a42a59294 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:36.099941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e17f72c8-b5f2-4a66-8bfa-a27a42a59294 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:36.254516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e17f72c8-b5f2-4a66-8bfa-a27a42a59294 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:36.263104 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e17f72c8-b5f2-4a66-8bfa-a27a42a59294 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.176945 Aug 07 01:36:36.263717 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1114/1996] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:36 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:37.297260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8f9d9e47-6012-4c71-948c-b1784ec6cda1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:37.297965 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8f9d9e47-6012-4c71-948c-b1784ec6cda1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:37.309021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f9d9e47-6012-4c71-948c-b1784ec6cda1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:37.309891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f9d9e47-6012-4c71-948c-b1784ec6cda1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:37.310522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f9d9e47-6012-4c71-948c-b1784ec6cda1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:37.311313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f9d9e47-6012-4c71-948c-b1784ec6cda1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:37.311824 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f9d9e47-6012-4c71-948c-b1784ec6cda1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:37.312347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8f9d9e47-6012-4c71-948c-b1784ec6cda1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:37.478074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8f9d9e47-6012-4c71-948c-b1784ec6cda1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:37.487321 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8f9d9e47-6012-4c71-948c-b1784ec6cda1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.194405 Aug 07 01:36:37.488058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1115/1997] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:37 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:38.511789 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-15d54d5a-740d-4be2-8b1b-219277997c24 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:38.512627 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-15d54d5a-740d-4be2-8b1b-219277997c24 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:38.519879 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15d54d5a-740d-4be2-8b1b-219277997c24 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:38.520490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15d54d5a-740d-4be2-8b1b-219277997c24 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:38.520957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15d54d5a-740d-4be2-8b1b-219277997c24 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:38.521749 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15d54d5a-740d-4be2-8b1b-219277997c24 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:38.522236 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15d54d5a-740d-4be2-8b1b-219277997c24 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:38.522908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15d54d5a-740d-4be2-8b1b-219277997c24 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:38.679561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-15d54d5a-740d-4be2-8b1b-219277997c24 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:38.688707 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-15d54d5a-740d-4be2-8b1b-219277997c24 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.181214 Aug 07 01:36:38.689458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1116/1998] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:38 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:39.711074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-04b8ed17-7f7f-42b9-b31c-51af63d49037 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:39.711774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-04b8ed17-7f7f-42b9-b31c-51af63d49037 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:39.718911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04b8ed17-7f7f-42b9-b31c-51af63d49037 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:39.719410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04b8ed17-7f7f-42b9-b31c-51af63d49037 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:39.719857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04b8ed17-7f7f-42b9-b31c-51af63d49037 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:39.720613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04b8ed17-7f7f-42b9-b31c-51af63d49037 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:39.721076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04b8ed17-7f7f-42b9-b31c-51af63d49037 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:39.721507 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-04b8ed17-7f7f-42b9-b31c-51af63d49037 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:39.871792 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-04b8ed17-7f7f-42b9-b31c-51af63d49037 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:39.883525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-04b8ed17-7f7f-42b9-b31c-51af63d49037 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.176386 Aug 07 01:36:39.884516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1117/1999] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:39 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:40.907082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3f2b0347-b92c-44f9-8ed0-f1ff02f10ead tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:40.907823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3f2b0347-b92c-44f9-8ed0-f1ff02f10ead tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:40.914631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f2b0347-b92c-44f9-8ed0-f1ff02f10ead tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:40.915110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f2b0347-b92c-44f9-8ed0-f1ff02f10ead tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:40.915548 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f2b0347-b92c-44f9-8ed0-f1ff02f10ead tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:40.916842 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f2b0347-b92c-44f9-8ed0-f1ff02f10ead tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:40.917298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f2b0347-b92c-44f9-8ed0-f1ff02f10ead tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:40.917725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3f2b0347-b92c-44f9-8ed0-f1ff02f10ead tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:41.088842 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3f2b0347-b92c-44f9-8ed0-f1ff02f10ead tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:41.097971 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3f2b0347-b92c-44f9-8ed0-f1ff02f10ead tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.194743 Aug 07 01:36:41.098572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1118/2000] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:40 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:42.132032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-28f6cd64-c488-421f-89c8-b3ea37810ae6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:42.133185 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-28f6cd64-c488-421f-89c8-b3ea37810ae6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:42.140946 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-28f6cd64-c488-421f-89c8-b3ea37810ae6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:42.141634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-28f6cd64-c488-421f-89c8-b3ea37810ae6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:42.142185 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-28f6cd64-c488-421f-89c8-b3ea37810ae6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:42.143172 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-28f6cd64-c488-421f-89c8-b3ea37810ae6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:42.143645 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-28f6cd64-c488-421f-89c8-b3ea37810ae6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:42.144084 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-28f6cd64-c488-421f-89c8-b3ea37810ae6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:42.305105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-28f6cd64-c488-421f-89c8-b3ea37810ae6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:42.314471 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-28f6cd64-c488-421f-89c8-b3ea37810ae6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.186898 Aug 07 01:36:42.315057 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1119/2001] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:42 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:43.337610 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-94955f40-bc45-459f-a272-10919c010e73 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:43.338821 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-94955f40-bc45-459f-a272-10919c010e73 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:43.345618 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94955f40-bc45-459f-a272-10919c010e73 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:43.346118 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94955f40-bc45-459f-a272-10919c010e73 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:43.346553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94955f40-bc45-459f-a272-10919c010e73 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:43.347262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94955f40-bc45-459f-a272-10919c010e73 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:43.347712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94955f40-bc45-459f-a272-10919c010e73 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:43.348132 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94955f40-bc45-459f-a272-10919c010e73 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:43.499849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-94955f40-bc45-459f-a272-10919c010e73 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:43.509166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-94955f40-bc45-459f-a272-10919c010e73 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.175587 Aug 07 01:36:43.510338 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1120/2002] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:43 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:44.531915 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e0ba782e-f256-4d3b-ac92-e18172d369ff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:44.532676 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e0ba782e-f256-4d3b-ac92-e18172d369ff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:44.539563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ba782e-f256-4d3b-ac92-e18172d369ff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:44.540108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ba782e-f256-4d3b-ac92-e18172d369ff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:44.540614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ba782e-f256-4d3b-ac92-e18172d369ff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:44.541455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ba782e-f256-4d3b-ac92-e18172d369ff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:44.541960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ba782e-f256-4d3b-ac92-e18172d369ff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:44.542463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ba782e-f256-4d3b-ac92-e18172d369ff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:44.702205 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e0ba782e-f256-4d3b-ac92-e18172d369ff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:44.711644 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e0ba782e-f256-4d3b-ac92-e18172d369ff tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.183896 Aug 07 01:36:44.712396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1121/2003] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:44 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:45.734253 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-53e0a153-900f-4916-8176-b9fb108532af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:45.734942 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-53e0a153-900f-4916-8176-b9fb108532af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:45.741810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53e0a153-900f-4916-8176-b9fb108532af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:45.742311 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53e0a153-900f-4916-8176-b9fb108532af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:45.742742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53e0a153-900f-4916-8176-b9fb108532af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:45.743454 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53e0a153-900f-4916-8176-b9fb108532af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:45.743907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53e0a153-900f-4916-8176-b9fb108532af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:45.744362 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53e0a153-900f-4916-8176-b9fb108532af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:45.901635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-53e0a153-900f-4916-8176-b9fb108532af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:45.904113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:36:45.911674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-53e0a153-900f-4916-8176-b9fb108532af tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.181680 Aug 07 01:36:45.912288 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1122/2004] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:45 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:46.937010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d5e3c55f-7169-45d7-872b-8c6a3a787e60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:46.937723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d5e3c55f-7169-45d7-872b-8c6a3a787e60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:46.949635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d5e3c55f-7169-45d7-872b-8c6a3a787e60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:46.950493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d5e3c55f-7169-45d7-872b-8c6a3a787e60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:46.951256 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d5e3c55f-7169-45d7-872b-8c6a3a787e60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:46.952511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d5e3c55f-7169-45d7-872b-8c6a3a787e60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:46.953341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d5e3c55f-7169-45d7-872b-8c6a3a787e60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:46.954103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d5e3c55f-7169-45d7-872b-8c6a3a787e60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:47.130396 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d5e3c55f-7169-45d7-872b-8c6a3a787e60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:47.140084 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d5e3c55f-7169-45d7-872b-8c6a3a787e60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.210315 Aug 07 01:36:47.140720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1123/2005] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:46 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:36:48.162995 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-95dd9aa9-7c26-4061-83c1-b21dd07c5a7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:48.163705 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-95dd9aa9-7c26-4061-83c1-b21dd07c5a7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:48.170962 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95dd9aa9-7c26-4061-83c1-b21dd07c5a7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:48.171453 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95dd9aa9-7c26-4061-83c1-b21dd07c5a7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:48.171895 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95dd9aa9-7c26-4061-83c1-b21dd07c5a7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:48.172726 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95dd9aa9-7c26-4061-83c1-b21dd07c5a7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:48.173188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95dd9aa9-7c26-4061-83c1-b21dd07c5a7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:48.173619 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-95dd9aa9-7c26-4061-83c1-b21dd07c5a7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:48.323571 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-95dd9aa9-7c26-4061-83c1-b21dd07c5a7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:48.332419 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-95dd9aa9-7c26-4061-83c1-b21dd07c5a7a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.173253 Aug 07 01:36:48.333030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1124/2006] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:48 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:36:49.355342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5538de64-acd6-4bda-8cb1-4cc193b44f4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:49.356091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5538de64-acd6-4bda-8cb1-4cc193b44f4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:49.363546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5538de64-acd6-4bda-8cb1-4cc193b44f4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:49.364103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5538de64-acd6-4bda-8cb1-4cc193b44f4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:49.364634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5538de64-acd6-4bda-8cb1-4cc193b44f4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:49.365444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5538de64-acd6-4bda-8cb1-4cc193b44f4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:49.365999 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5538de64-acd6-4bda-8cb1-4cc193b44f4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:49.366543 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5538de64-acd6-4bda-8cb1-4cc193b44f4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:49.538869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5538de64-acd6-4bda-8cb1-4cc193b44f4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:49.547760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5538de64-acd6-4bda-8cb1-4cc193b44f4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.196383 Aug 07 01:36:49.548411 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1125/2007] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:49 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:50.569698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c4ead79b-1a6b-4697-b762-f74a63923593 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:50.570633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-c4ead79b-1a6b-4697-b762-f74a63923593 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:50.577554 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c4ead79b-1a6b-4697-b762-f74a63923593 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:50.578038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c4ead79b-1a6b-4697-b762-f74a63923593 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:50.578475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c4ead79b-1a6b-4697-b762-f74a63923593 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:50.579184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c4ead79b-1a6b-4697-b762-f74a63923593 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:50.579640 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c4ead79b-1a6b-4697-b762-f74a63923593 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:50.580071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-c4ead79b-1a6b-4697-b762-f74a63923593 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:50.764615 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-c4ead79b-1a6b-4697-b762-f74a63923593 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:50.773878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c4ead79b-1a6b-4697-b762-f74a63923593 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.208151 Aug 07 01:36:50.774488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1126/2008] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:50 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:51.795893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3737a54f-dc64-4b2b-812e-512a628f1120 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:51.796718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3737a54f-dc64-4b2b-812e-512a628f1120 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:51.803618 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3737a54f-dc64-4b2b-812e-512a628f1120 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:51.804118 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3737a54f-dc64-4b2b-812e-512a628f1120 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:51.804593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3737a54f-dc64-4b2b-812e-512a628f1120 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:51.805319 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3737a54f-dc64-4b2b-812e-512a628f1120 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:51.805784 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3737a54f-dc64-4b2b-812e-512a628f1120 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:51.806211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3737a54f-dc64-4b2b-812e-512a628f1120 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:51.963756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3737a54f-dc64-4b2b-812e-512a628f1120 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:51.974057 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3737a54f-dc64-4b2b-812e-512a628f1120 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.181970 Aug 07 01:36:51.974746 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1127/2009] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:51 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:52.996497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-56c0cf6a-a849-47ea-9c8f-7e3b737ac7c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:52.997349 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-56c0cf6a-a849-47ea-9c8f-7e3b737ac7c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:53.003861 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-56c0cf6a-a849-47ea-9c8f-7e3b737ac7c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:53.004392 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-56c0cf6a-a849-47ea-9c8f-7e3b737ac7c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:53.004838 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-56c0cf6a-a849-47ea-9c8f-7e3b737ac7c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:53.005550 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-56c0cf6a-a849-47ea-9c8f-7e3b737ac7c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:53.006006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-56c0cf6a-a849-47ea-9c8f-7e3b737ac7c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:53.006429 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-56c0cf6a-a849-47ea-9c8f-7e3b737ac7c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:53.191718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-56c0cf6a-a849-47ea-9c8f-7e3b737ac7c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:53.201058 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-56c0cf6a-a849-47ea-9c8f-7e3b737ac7c7 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.208329 Aug 07 01:36:53.201675 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1128/2010] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:52 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:54.236116 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fd77f2cd-3626-4ed3-9b99-7d72341c18fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:54.236937 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fd77f2cd-3626-4ed3-9b99-7d72341c18fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:54.245181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd77f2cd-3626-4ed3-9b99-7d72341c18fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:54.245679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd77f2cd-3626-4ed3-9b99-7d72341c18fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:54.246271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd77f2cd-3626-4ed3-9b99-7d72341c18fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:54.247013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd77f2cd-3626-4ed3-9b99-7d72341c18fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:54.247508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd77f2cd-3626-4ed3-9b99-7d72341c18fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:54.247936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fd77f2cd-3626-4ed3-9b99-7d72341c18fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:54.422458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-fd77f2cd-3626-4ed3-9b99-7d72341c18fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:54.438508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fd77f2cd-3626-4ed3-9b99-7d72341c18fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.206221 Aug 07 01:36:54.439776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1129/2011] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:54 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:55.475284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-94593255-86e8-4ffb-92ec-7506622c7b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:55.476022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-94593255-86e8-4ffb-92ec-7506622c7b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:55.483107 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94593255-86e8-4ffb-92ec-7506622c7b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:55.483607 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94593255-86e8-4ffb-92ec-7506622c7b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:55.484038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94593255-86e8-4ffb-92ec-7506622c7b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:55.484779 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94593255-86e8-4ffb-92ec-7506622c7b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:55.485228 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94593255-86e8-4ffb-92ec-7506622c7b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:55.485656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-94593255-86e8-4ffb-92ec-7506622c7b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:55.639539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-94593255-86e8-4ffb-92ec-7506622c7b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:55.648879 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-94593255-86e8-4ffb-92ec-7506622c7b79 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.177292 Aug 07 01:36:55.649987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1130/2012] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:55 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:36:56.671879 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-330226f4-4ed6-4e8b-a150-8db7d2a2047f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:56.672777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-330226f4-4ed6-4e8b-a150-8db7d2a2047f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:56.680199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-330226f4-4ed6-4e8b-a150-8db7d2a2047f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:56.680793 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-330226f4-4ed6-4e8b-a150-8db7d2a2047f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:56.681277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-330226f4-4ed6-4e8b-a150-8db7d2a2047f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:56.682055 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-330226f4-4ed6-4e8b-a150-8db7d2a2047f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:56.682562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-330226f4-4ed6-4e8b-a150-8db7d2a2047f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:56.683035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-330226f4-4ed6-4e8b-a150-8db7d2a2047f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:56.832973 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-330226f4-4ed6-4e8b-a150-8db7d2a2047f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:56.842399 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-330226f4-4ed6-4e8b-a150-8db7d2a2047f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.174397 Aug 07 01:36:56.842997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1131/2013] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:56 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:57.864380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d08efb63-bd42-40fe-abb6-3be89960886a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:57.865161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d08efb63-bd42-40fe-abb6-3be89960886a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:57.872807 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d08efb63-bd42-40fe-abb6-3be89960886a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:57.873437 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d08efb63-bd42-40fe-abb6-3be89960886a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:57.874079 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d08efb63-bd42-40fe-abb6-3be89960886a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:57.874969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d08efb63-bd42-40fe-abb6-3be89960886a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:57.875529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d08efb63-bd42-40fe-abb6-3be89960886a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:57.876010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d08efb63-bd42-40fe-abb6-3be89960886a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:58.040096 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d08efb63-bd42-40fe-abb6-3be89960886a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:58.055046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d08efb63-bd42-40fe-abb6-3be89960886a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.194633 Aug 07 01:36:58.055849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1132/2014] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:57 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:36:59.078062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b14c3d0c-5f29-4086-af4b-6f6f4c8943cd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:36:59.078839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b14c3d0c-5f29-4086-af4b-6f6f4c8943cd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:36:59.085916 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b14c3d0c-5f29-4086-af4b-6f6f4c8943cd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:59.086410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b14c3d0c-5f29-4086-af4b-6f6f4c8943cd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:59.086853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b14c3d0c-5f29-4086-af4b-6f6f4c8943cd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:59.087578 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b14c3d0c-5f29-4086-af4b-6f6f4c8943cd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:36:59.088634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b14c3d0c-5f29-4086-af4b-6f6f4c8943cd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:36:59.089063 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b14c3d0c-5f29-4086-af4b-6f6f4c8943cd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:36:59.235750 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b14c3d0c-5f29-4086-af4b-6f6f4c8943cd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:36:59.244625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b14c3d0c-5f29-4086-af4b-6f6f4c8943cd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.170463 Aug 07 01:36:59.245226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1133/2015] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:36:59 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:00.267359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8a19c3c9-a2e1-4a82-baf5-96566ca38894 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:00.268104 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8a19c3c9-a2e1-4a82-baf5-96566ca38894 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:00.275741 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a19c3c9-a2e1-4a82-baf5-96566ca38894 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:00.276267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a19c3c9-a2e1-4a82-baf5-96566ca38894 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:00.276723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a19c3c9-a2e1-4a82-baf5-96566ca38894 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:00.277496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a19c3c9-a2e1-4a82-baf5-96566ca38894 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:00.277897 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a19c3c9-a2e1-4a82-baf5-96566ca38894 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:00.278400 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a19c3c9-a2e1-4a82-baf5-96566ca38894 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:00.451288 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8a19c3c9-a2e1-4a82-baf5-96566ca38894 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:00.461711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8a19c3c9-a2e1-4a82-baf5-96566ca38894 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.198056 Aug 07 01:37:00.464324 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1134/2016] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:00 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:01.485372 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5b34daae-8843-441d-8d4b-ace899fa6e3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:01.486102 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5b34daae-8843-441d-8d4b-ace899fa6e3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:01.493490 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b34daae-8843-441d-8d4b-ace899fa6e3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:01.494060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b34daae-8843-441d-8d4b-ace899fa6e3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:01.494501 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b34daae-8843-441d-8d4b-ace899fa6e3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:01.495228 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b34daae-8843-441d-8d4b-ace899fa6e3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:01.495683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b34daae-8843-441d-8d4b-ace899fa6e3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:01.496111 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5b34daae-8843-441d-8d4b-ace899fa6e3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:01.648801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5b34daae-8843-441d-8d4b-ace899fa6e3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:01.658175 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5b34daae-8843-441d-8d4b-ace899fa6e3f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.176639 Aug 07 01:37:01.658784 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1135/2017] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:01 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:02.681713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-74eeb4f8-6635-422f-9687-a52080b7526b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:02.682529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-74eeb4f8-6635-422f-9687-a52080b7526b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:02.689648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-74eeb4f8-6635-422f-9687-a52080b7526b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:02.690181 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-74eeb4f8-6635-422f-9687-a52080b7526b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:02.690629 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-74eeb4f8-6635-422f-9687-a52080b7526b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:02.691442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-74eeb4f8-6635-422f-9687-a52080b7526b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:02.691904 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-74eeb4f8-6635-422f-9687-a52080b7526b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:02.692471 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-74eeb4f8-6635-422f-9687-a52080b7526b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:02.874099 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-74eeb4f8-6635-422f-9687-a52080b7526b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:02.882867 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-74eeb4f8-6635-422f-9687-a52080b7526b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.205958 Aug 07 01:37:02.883504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1136/2018] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:02 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:03.904872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-18f0d66f-adb7-4a73-b95b-8879c2efcc8d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:03.905922 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-18f0d66f-adb7-4a73-b95b-8879c2efcc8d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:03.907906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=50 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:37:03.913791 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-18f0d66f-adb7-4a73-b95b-8879c2efcc8d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:03.914295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-18f0d66f-adb7-4a73-b95b-8879c2efcc8d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:03.914742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-18f0d66f-adb7-4a73-b95b-8879c2efcc8d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:03.915475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-18f0d66f-adb7-4a73-b95b-8879c2efcc8d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:03.916609 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-18f0d66f-adb7-4a73-b95b-8879c2efcc8d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:03.917049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-18f0d66f-adb7-4a73-b95b-8879c2efcc8d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:04.081264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-18f0d66f-adb7-4a73-b95b-8879c2efcc8d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:04.096355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-18f0d66f-adb7-4a73-b95b-8879c2efcc8d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.195892 Aug 07 01:37:04.097402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1137/2019] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:03 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:05.132860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-2762ba0d-821e-4e2f-a025-026fd1469214 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:05.133943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-2762ba0d-821e-4e2f-a025-026fd1469214 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:05.142410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2762ba0d-821e-4e2f-a025-026fd1469214 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:05.142970 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2762ba0d-821e-4e2f-a025-026fd1469214 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:05.143458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2762ba0d-821e-4e2f-a025-026fd1469214 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:05.144346 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2762ba0d-821e-4e2f-a025-026fd1469214 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:05.144918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2762ba0d-821e-4e2f-a025-026fd1469214 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:05.145437 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-2762ba0d-821e-4e2f-a025-026fd1469214 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:05.301731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-2762ba0d-821e-4e2f-a025-026fd1469214 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:05.312501 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-2762ba0d-821e-4e2f-a025-026fd1469214 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.183651 Aug 07 01:37:05.313153 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1138/2020] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:05 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:06.334870 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9084fde4-0ed9-41b0-8d83-d5c96a145661 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:06.335676 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9084fde4-0ed9-41b0-8d83-d5c96a145661 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:06.343747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9084fde4-0ed9-41b0-8d83-d5c96a145661 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:06.344342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9084fde4-0ed9-41b0-8d83-d5c96a145661 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:06.344842 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9084fde4-0ed9-41b0-8d83-d5c96a145661 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:06.345611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9084fde4-0ed9-41b0-8d83-d5c96a145661 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:06.346128 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9084fde4-0ed9-41b0-8d83-d5c96a145661 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:06.346613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9084fde4-0ed9-41b0-8d83-d5c96a145661 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:06.362867 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:37:06.505736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9084fde4-0ed9-41b0-8d83-d5c96a145661 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:06.515373 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9084fde4-0ed9-41b0-8d83-d5c96a145661 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.184539 Aug 07 01:37:06.515964 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1139/2021] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:06 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:07.537184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0b7fc94b-b1ea-46d2-a8c5-d1edd3135ccc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:07.538013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0b7fc94b-b1ea-46d2-a8c5-d1edd3135ccc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:07.545506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b7fc94b-b1ea-46d2-a8c5-d1edd3135ccc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:07.546144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b7fc94b-b1ea-46d2-a8c5-d1edd3135ccc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:07.546678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b7fc94b-b1ea-46d2-a8c5-d1edd3135ccc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:07.547428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b7fc94b-b1ea-46d2-a8c5-d1edd3135ccc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:07.547890 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b7fc94b-b1ea-46d2-a8c5-d1edd3135ccc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:07.548364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b7fc94b-b1ea-46d2-a8c5-d1edd3135ccc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:07.729592 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0b7fc94b-b1ea-46d2-a8c5-d1edd3135ccc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:07.737921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0b7fc94b-b1ea-46d2-a8c5-d1edd3135ccc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.205207 Aug 07 01:37:07.738511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1140/2022] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:07 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:08.463566 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:37:08.464193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:37:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-volumes (ip 10.222.0.26) !!! Aug 07 01:37:08.760560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-cbf468e7-0c1e-400d-8425-ed40d8094bcb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:08.761188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-cbf468e7-0c1e-400d-8425-ed40d8094bcb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:08.767855 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cbf468e7-0c1e-400d-8425-ed40d8094bcb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:08.768380 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cbf468e7-0c1e-400d-8425-ed40d8094bcb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:08.768827 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cbf468e7-0c1e-400d-8425-ed40d8094bcb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:08.769557 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cbf468e7-0c1e-400d-8425-ed40d8094bcb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:08.770007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cbf468e7-0c1e-400d-8425-ed40d8094bcb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:08.770430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-cbf468e7-0c1e-400d-8425-ed40d8094bcb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:08.923898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-cbf468e7-0c1e-400d-8425-ed40d8094bcb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:08.933693 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-cbf468e7-0c1e-400d-8425-ed40d8094bcb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.177620 Aug 07 01:37:08.934304 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1141/2023] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:08 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:09.955538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0d1df163-d974-4e54-aa70-ec5121abf17d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:09.956472 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0d1df163-d974-4e54-aa70-ec5121abf17d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:09.964422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d1df163-d974-4e54-aa70-ec5121abf17d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:09.964944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d1df163-d974-4e54-aa70-ec5121abf17d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:09.965466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d1df163-d974-4e54-aa70-ec5121abf17d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:09.966977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d1df163-d974-4e54-aa70-ec5121abf17d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:09.967776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d1df163-d974-4e54-aa70-ec5121abf17d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:09.968790 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d1df163-d974-4e54-aa70-ec5121abf17d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:10.120171 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0d1df163-d974-4e54-aa70-ec5121abf17d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:10.130232 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0d1df163-d974-4e54-aa70-ec5121abf17d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.178849 Aug 07 01:37:10.130827 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1142/2024] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:09 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:11.154409 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6c8dfbdd-6c24-46ae-bbdb-5d99bc2e701a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:11.155197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6c8dfbdd-6c24-46ae-bbdb-5d99bc2e701a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:11.162398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c8dfbdd-6c24-46ae-bbdb-5d99bc2e701a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:11.162946 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c8dfbdd-6c24-46ae-bbdb-5d99bc2e701a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:11.163509 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c8dfbdd-6c24-46ae-bbdb-5d99bc2e701a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:11.164316 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c8dfbdd-6c24-46ae-bbdb-5d99bc2e701a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:11.164858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c8dfbdd-6c24-46ae-bbdb-5d99bc2e701a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:11.165339 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6c8dfbdd-6c24-46ae-bbdb-5d99bc2e701a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:11.324086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6c8dfbdd-6c24-46ae-bbdb-5d99bc2e701a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:11.333223 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6c8dfbdd-6c24-46ae-bbdb-5d99bc2e701a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.183069 Aug 07 01:37:11.333836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1143/2025] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:11 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:12.355859 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9a8813e8-d93d-4017-8927-713f6bc0c0f6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:12.356766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9a8813e8-d93d-4017-8927-713f6bc0c0f6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:12.364658 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a8813e8-d93d-4017-8927-713f6bc0c0f6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:12.365234 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a8813e8-d93d-4017-8927-713f6bc0c0f6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:12.365812 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a8813e8-d93d-4017-8927-713f6bc0c0f6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:12.366612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a8813e8-d93d-4017-8927-713f6bc0c0f6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:12.367127 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a8813e8-d93d-4017-8927-713f6bc0c0f6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:12.367631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9a8813e8-d93d-4017-8927-713f6bc0c0f6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:12.535494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9a8813e8-d93d-4017-8927-713f6bc0c0f6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:12.553686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9a8813e8-d93d-4017-8927-713f6bc0c0f6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.202157 Aug 07 01:37:12.554478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1144/2026] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:12 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:13.577199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9e792434-8988-4003-b467-6b7d424dd896 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:13.577990 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9e792434-8988-4003-b467-6b7d424dd896 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:13.586313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e792434-8988-4003-b467-6b7d424dd896 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:13.586909 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e792434-8988-4003-b467-6b7d424dd896 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:13.587447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e792434-8988-4003-b467-6b7d424dd896 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:13.588289 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e792434-8988-4003-b467-6b7d424dd896 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:13.588848 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e792434-8988-4003-b467-6b7d424dd896 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:13.589349 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9e792434-8988-4003-b467-6b7d424dd896 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:13.750389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9e792434-8988-4003-b467-6b7d424dd896 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:13.759721 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9e792434-8988-4003-b467-6b7d424dd896 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.186648 Aug 07 01:37:13.760456 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1145/2027] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:13 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:14.782079 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0b471cb4-8f6a-4e61-9756-81846d7d1a4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:14.782786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0b471cb4-8f6a-4e61-9756-81846d7d1a4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:14.789862 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b471cb4-8f6a-4e61-9756-81846d7d1a4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:14.790354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b471cb4-8f6a-4e61-9756-81846d7d1a4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:14.791275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b471cb4-8f6a-4e61-9756-81846d7d1a4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:14.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b471cb4-8f6a-4e61-9756-81846d7d1a4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:14.792523 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b471cb4-8f6a-4e61-9756-81846d7d1a4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:14.792947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0b471cb4-8f6a-4e61-9756-81846d7d1a4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:14.972822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0b471cb4-8f6a-4e61-9756-81846d7d1a4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:14.988447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0b471cb4-8f6a-4e61-9756-81846d7d1a4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.209840 Aug 07 01:37:14.989437 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1146/2028] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:14 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:16.016228 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f2ad9111-c9e8-4c14-b7a9-5d77f28b0e11 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:16.017079 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f2ad9111-c9e8-4c14-b7a9-5d77f28b0e11 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:16.023701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2ad9111-c9e8-4c14-b7a9-5d77f28b0e11 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:16.024195 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2ad9111-c9e8-4c14-b7a9-5d77f28b0e11 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:16.024662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2ad9111-c9e8-4c14-b7a9-5d77f28b0e11 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:16.025426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2ad9111-c9e8-4c14-b7a9-5d77f28b0e11 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:16.025887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2ad9111-c9e8-4c14-b7a9-5d77f28b0e11 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:16.026313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2ad9111-c9e8-4c14-b7a9-5d77f28b0e11 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:16.175754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f2ad9111-c9e8-4c14-b7a9-5d77f28b0e11 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:16.186005 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f2ad9111-c9e8-4c14-b7a9-5d77f28b0e11 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.173850 Aug 07 01:37:16.186603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1147/2029] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:16 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:17.209157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ef225f33-d4f6-449d-93d0-ae702cfd4cab tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:17.210174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ef225f33-d4f6-449d-93d0-ae702cfd4cab tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:17.219911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef225f33-d4f6-449d-93d0-ae702cfd4cab tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:17.220813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef225f33-d4f6-449d-93d0-ae702cfd4cab tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:17.221580 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef225f33-d4f6-449d-93d0-ae702cfd4cab tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:17.222834 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef225f33-d4f6-449d-93d0-ae702cfd4cab tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:17.223623 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef225f33-d4f6-449d-93d0-ae702cfd4cab tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:17.224402 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ef225f33-d4f6-449d-93d0-ae702cfd4cab tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:17.381440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ef225f33-d4f6-449d-93d0-ae702cfd4cab tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:17.390539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ef225f33-d4f6-449d-93d0-ae702cfd4cab tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.185591 Aug 07 01:37:17.391148 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1148/2030] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:17 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:18.413604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1a7acb66-4009-421e-9b78-5cfcb54939fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:18.414335 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1a7acb66-4009-421e-9b78-5cfcb54939fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:18.421366 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a7acb66-4009-421e-9b78-5cfcb54939fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:18.421865 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a7acb66-4009-421e-9b78-5cfcb54939fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:18.422337 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a7acb66-4009-421e-9b78-5cfcb54939fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:18.423077 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a7acb66-4009-421e-9b78-5cfcb54939fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:18.423541 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a7acb66-4009-421e-9b78-5cfcb54939fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:18.423979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1a7acb66-4009-421e-9b78-5cfcb54939fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:18.580477 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1a7acb66-4009-421e-9b78-5cfcb54939fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:18.590563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1a7acb66-4009-421e-9b78-5cfcb54939fe tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.180911 Aug 07 01:37:18.591176 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1149/2031] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:18 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:19.611551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a420f89f-11d7-44c6-a963-db791ce15f3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:19.612505 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a420f89f-11d7-44c6-a963-db791ce15f3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:19.619345 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a420f89f-11d7-44c6-a963-db791ce15f3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:19.619846 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a420f89f-11d7-44c6-a963-db791ce15f3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:19.620314 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a420f89f-11d7-44c6-a963-db791ce15f3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:19.621026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a420f89f-11d7-44c6-a963-db791ce15f3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:19.621486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a420f89f-11d7-44c6-a963-db791ce15f3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:19.621918 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a420f89f-11d7-44c6-a963-db791ce15f3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:19.787225 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a420f89f-11d7-44c6-a963-db791ce15f3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:19.796945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a420f89f-11d7-44c6-a963-db791ce15f3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.189533 Aug 07 01:37:19.797535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1150/2032] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:19 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:20.822440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0f42e76c-c1c6-4213-aa52-2cc88c198e51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:20.823504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0f42e76c-c1c6-4213-aa52-2cc88c198e51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:20.833907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f42e76c-c1c6-4213-aa52-2cc88c198e51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:20.835030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f42e76c-c1c6-4213-aa52-2cc88c198e51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:20.835775 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f42e76c-c1c6-4213-aa52-2cc88c198e51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:20.838141 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f42e76c-c1c6-4213-aa52-2cc88c198e51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:20.839334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f42e76c-c1c6-4213-aa52-2cc88c198e51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:20.840936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0f42e76c-c1c6-4213-aa52-2cc88c198e51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:21.128092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0f42e76c-c1c6-4213-aa52-2cc88c198e51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:21.144508 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0f42e76c-c1c6-4213-aa52-2cc88c198e51 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1560 microversion: 2.1 time: 0.329143 Aug 07 01:37:21.146062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1151/2033] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:20 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1560 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:21.600275 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d8a5ac91-a36a-4da3-b16d-ebed1c746ce0 tempest-VolumesGetTestJSON-1001858442 tempest-VolumesGetTestJSON-1001858442-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 200 len: 352 microversion: 2.1 time: 111.578362 Aug 07 01:37:21.601736 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 883/2034] 10.222.0.26 () {60 vars in 1156 bytes} [Sun Aug 7 01:35:30 2022] POST /compute/v2.1/os-volumes => generated 352 bytes in 111581 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Aug 07 01:37:22.201180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-215e0555-cd8c-460d-9506-8b56c8a43a5c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:22.203678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-215e0555-cd8c-460d-9506-8b56c8a43a5c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:22.219940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-215e0555-cd8c-460d-9506-8b56c8a43a5c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:22.221929 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-215e0555-cd8c-460d-9506-8b56c8a43a5c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:22.223040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-215e0555-cd8c-460d-9506-8b56c8a43a5c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:22.224744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-215e0555-cd8c-460d-9506-8b56c8a43a5c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:22.225908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-215e0555-cd8c-460d-9506-8b56c8a43a5c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:22.227268 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-215e0555-cd8c-460d-9506-8b56c8a43a5c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:22.392919 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-215e0555-cd8c-460d-9506-8b56c8a43a5c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:22.406183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-215e0555-cd8c-460d-9506-8b56c8a43a5c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1518 microversion: 2.1 time: 0.231604 Aug 07 01:37:22.407295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1152/2035] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:22 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1518 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:22.552773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-a4f688fc-a896-49d3-b566-70e6338864cc req-86d1c7df-80c0-4438-97a5-a2a9552210a0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f481f4cc-2803-4f12-9cd2-db6055d1488c", "name": "network-vif-deleted", "tag": "77812789-b59f-4abe-9e20-b16f511c9fe2"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:37:22.563218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-a4f688fc-a896-49d3-b566-70e6338864cc req-86d1c7df-80c0-4438-97a5-a2a9552210a0 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:22.563690 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-a4f688fc-a896-49d3-b566-70e6338864cc req-86d1c7df-80c0-4438-97a5-a2a9552210a0 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:22.564113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-a4f688fc-a896-49d3-b566-70e6338864cc req-86d1c7df-80c0-4438-97a5-a2a9552210a0 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:22.577537 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-a4f688fc-a896-49d3-b566-70e6338864cc req-86d1c7df-80c0-4438-97a5-a2a9552210a0 service nova] Creating event network-vif-deleted:77812789-b59f-4abe-9e20-b16f511c9fe2 for instance f481f4cc-2803-4f12-9cd2-db6055d1488c on hv-cinder-maste Aug 07 01:37:22.583902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-a4f688fc-a896-49d3-b566-70e6338864cc req-86d1c7df-80c0-4438-97a5-a2a9552210a0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.234228 Aug 07 01:37:22.584801 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 884/2036] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:37:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:37:23.442806 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-735e4ef4-e5aa-4ae6-9267-a867c8b6af19 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:23.443513 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-735e4ef4-e5aa-4ae6-9267-a867c8b6af19 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:23.451150 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735e4ef4-e5aa-4ae6-9267-a867c8b6af19 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:23.451633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735e4ef4-e5aa-4ae6-9267-a867c8b6af19 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:23.452074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735e4ef4-e5aa-4ae6-9267-a867c8b6af19 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:23.452877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735e4ef4-e5aa-4ae6-9267-a867c8b6af19 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:23.453361 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735e4ef4-e5aa-4ae6-9267-a867c8b6af19 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:23.453801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-735e4ef4-e5aa-4ae6-9267-a867c8b6af19 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:23.538803 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-735e4ef4-e5aa-4ae6-9267-a867c8b6af19 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:23.549656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-735e4ef4-e5aa-4ae6-9267-a867c8b6af19 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.111342 Aug 07 01:37:23.550487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1153/2037] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:23 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:24.573909 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4d84faed-d87d-44ad-b3ca-d51a95b83325 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:24.574776 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4d84faed-d87d-44ad-b3ca-d51a95b83325 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:24.581949 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d84faed-d87d-44ad-b3ca-d51a95b83325 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:24.582441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d84faed-d87d-44ad-b3ca-d51a95b83325 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:24.582883 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d84faed-d87d-44ad-b3ca-d51a95b83325 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:24.583604 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d84faed-d87d-44ad-b3ca-d51a95b83325 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:24.584113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d84faed-d87d-44ad-b3ca-d51a95b83325 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:24.584826 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4d84faed-d87d-44ad-b3ca-d51a95b83325 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:24.674215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4d84faed-d87d-44ad-b3ca-d51a95b83325 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:24.683325 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4d84faed-d87d-44ad-b3ca-d51a95b83325 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.114384 Aug 07 01:37:24.683930 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 885/2038] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:24 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:25.705751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-21a8f0ca-5f27-4a53-b30a-06a7ebaa2bee tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:25.706424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-21a8f0ca-5f27-4a53-b30a-06a7ebaa2bee tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:25.714621 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-21a8f0ca-5f27-4a53-b30a-06a7ebaa2bee tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:25.715120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-21a8f0ca-5f27-4a53-b30a-06a7ebaa2bee tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:25.715557 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-21a8f0ca-5f27-4a53-b30a-06a7ebaa2bee tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:25.716304 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-21a8f0ca-5f27-4a53-b30a-06a7ebaa2bee tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:25.716760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-21a8f0ca-5f27-4a53-b30a-06a7ebaa2bee tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:25.717189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-21a8f0ca-5f27-4a53-b30a-06a7ebaa2bee tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:25.788736 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-21a8f0ca-5f27-4a53-b30a-06a7ebaa2bee tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:25.799260 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-21a8f0ca-5f27-4a53-b30a-06a7ebaa2bee tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.097553 Aug 07 01:37:25.799868 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1154/2039] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:25 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:26.821399 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1fd90ed7-6508-483a-8b1f-b0f398d37782 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:26.822690 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1fd90ed7-6508-483a-8b1f-b0f398d37782 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:26.830054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1fd90ed7-6508-483a-8b1f-b0f398d37782 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:26.830708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1fd90ed7-6508-483a-8b1f-b0f398d37782 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:26.831230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1fd90ed7-6508-483a-8b1f-b0f398d37782 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:26.831941 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1fd90ed7-6508-483a-8b1f-b0f398d37782 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:26.832509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1fd90ed7-6508-483a-8b1f-b0f398d37782 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:26.833000 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1fd90ed7-6508-483a-8b1f-b0f398d37782 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:26.906230 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1fd90ed7-6508-483a-8b1f-b0f398d37782 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:26.914963 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1fd90ed7-6508-483a-8b1f-b0f398d37782 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.097511 Aug 07 01:37:26.915709 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 886/2040] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:26 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:27.936786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-93862915-8338-441a-bb7c-be893fb0340e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:27.937462 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-93862915-8338-441a-bb7c-be893fb0340e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:27.944527 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-93862915-8338-441a-bb7c-be893fb0340e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:27.945082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-93862915-8338-441a-bb7c-be893fb0340e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:27.945575 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-93862915-8338-441a-bb7c-be893fb0340e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:27.946360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-93862915-8338-441a-bb7c-be893fb0340e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:27.946893 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-93862915-8338-441a-bb7c-be893fb0340e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:27.947370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-93862915-8338-441a-bb7c-be893fb0340e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:28.025980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-93862915-8338-441a-bb7c-be893fb0340e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:28.034683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-93862915-8338-441a-bb7c-be893fb0340e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.101899 Aug 07 01:37:28.035285 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1155/2041] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:27 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:29.059652 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f0cfeb5d-da03-4132-a344-31bc4129da7e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:29.060327 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f0cfeb5d-da03-4132-a344-31bc4129da7e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:29.067813 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0cfeb5d-da03-4132-a344-31bc4129da7e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:29.068440 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0cfeb5d-da03-4132-a344-31bc4129da7e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:29.068930 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0cfeb5d-da03-4132-a344-31bc4129da7e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:29.069695 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0cfeb5d-da03-4132-a344-31bc4129da7e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:29.070247 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0cfeb5d-da03-4132-a344-31bc4129da7e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:29.070676 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0cfeb5d-da03-4132-a344-31bc4129da7e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:29.139355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f0cfeb5d-da03-4132-a344-31bc4129da7e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:29.148806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f0cfeb5d-da03-4132-a344-31bc4129da7e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.094487 Aug 07 01:37:29.149621 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 887/2042] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:29 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:30.175275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3c2f38b8-b99f-4d4f-9f7b-ac6733389109 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:30.176011 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3c2f38b8-b99f-4d4f-9f7b-ac6733389109 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:30.184539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c2f38b8-b99f-4d4f-9f7b-ac6733389109 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:30.185163 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c2f38b8-b99f-4d4f-9f7b-ac6733389109 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:30.185645 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c2f38b8-b99f-4d4f-9f7b-ac6733389109 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:30.186411 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c2f38b8-b99f-4d4f-9f7b-ac6733389109 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:30.186987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c2f38b8-b99f-4d4f-9f7b-ac6733389109 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:30.187491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3c2f38b8-b99f-4d4f-9f7b-ac6733389109 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:30.190681 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=100 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:37:30.258998 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3c2f38b8-b99f-4d4f-9f7b-ac6733389109 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:30.267891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3c2f38b8-b99f-4d4f-9f7b-ac6733389109 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.097611 Aug 07 01:37:30.268698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1156/2043] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:30 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:31.297618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-452fcc6a-3293-4d95-a69d-d4c3983cb49b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:31.298647 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-452fcc6a-3293-4d95-a69d-d4c3983cb49b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:31.307119 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-452fcc6a-3293-4d95-a69d-d4c3983cb49b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:31.307628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-452fcc6a-3293-4d95-a69d-d4c3983cb49b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:31.308076 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-452fcc6a-3293-4d95-a69d-d4c3983cb49b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:31.308856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-452fcc6a-3293-4d95-a69d-d4c3983cb49b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:31.309324 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-452fcc6a-3293-4d95-a69d-d4c3983cb49b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:31.309803 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-452fcc6a-3293-4d95-a69d-d4c3983cb49b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:31.377439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-452fcc6a-3293-4d95-a69d-d4c3983cb49b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:31.386273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-452fcc6a-3293-4d95-a69d-d4c3983cb49b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.093653 Aug 07 01:37:31.386873 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 888/2044] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:31 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:32.409080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-46c6b698-d19f-48cb-a978-1153aa337dfd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:32.409764 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-46c6b698-d19f-48cb-a978-1153aa337dfd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:32.416789 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-46c6b698-d19f-48cb-a978-1153aa337dfd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:32.417277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-46c6b698-d19f-48cb-a978-1153aa337dfd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:32.417711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-46c6b698-d19f-48cb-a978-1153aa337dfd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:32.418428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-46c6b698-d19f-48cb-a978-1153aa337dfd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:32.418872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-46c6b698-d19f-48cb-a978-1153aa337dfd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:32.419294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-46c6b698-d19f-48cb-a978-1153aa337dfd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:32.487535 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-46c6b698-d19f-48cb-a978-1153aa337dfd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:32.498952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-46c6b698-d19f-48cb-a978-1153aa337dfd tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.093672 Aug 07 01:37:32.499879 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1157/2045] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:32 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:33.528331 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-75486186-62de-4323-a103-b9e5045236a1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:33.529478 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-75486186-62de-4323-a103-b9e5045236a1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:33.541574 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-75486186-62de-4323-a103-b9e5045236a1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:33.542317 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-75486186-62de-4323-a103-b9e5045236a1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:33.543071 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-75486186-62de-4323-a103-b9e5045236a1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:33.544228 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-75486186-62de-4323-a103-b9e5045236a1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:33.545056 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-75486186-62de-4323-a103-b9e5045236a1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:33.545821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-75486186-62de-4323-a103-b9e5045236a1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:33.619477 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-75486186-62de-4323-a103-b9e5045236a1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:33.628313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-75486186-62de-4323-a103-b9e5045236a1 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.107645 Aug 07 01:37:33.628919 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 889/2046] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:33 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:34.662463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b55e4744-36b9-4f51-9a6f-f940678c0f5b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:34.663262 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b55e4744-36b9-4f51-9a6f-f940678c0f5b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:34.666372 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=42 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:37:34.672025 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b55e4744-36b9-4f51-9a6f-f940678c0f5b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:34.672656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b55e4744-36b9-4f51-9a6f-f940678c0f5b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:34.673102 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b55e4744-36b9-4f51-9a6f-f940678c0f5b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:34.673829 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b55e4744-36b9-4f51-9a6f-f940678c0f5b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:34.674287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b55e4744-36b9-4f51-9a6f-f940678c0f5b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:34.674754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b55e4744-36b9-4f51-9a6f-f940678c0f5b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:34.745255 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b55e4744-36b9-4f51-9a6f-f940678c0f5b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:34.754203 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b55e4744-36b9-4f51-9a6f-f940678c0f5b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.095926 Aug 07 01:37:34.755028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1158/2047] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:34 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:35.777959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-23071166-3d47-425d-9449-446b9b56554c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:35.778600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-23071166-3d47-425d-9449-446b9b56554c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:35.785774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-23071166-3d47-425d-9449-446b9b56554c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:35.786269 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-23071166-3d47-425d-9449-446b9b56554c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:35.786708 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-23071166-3d47-425d-9449-446b9b56554c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:35.787433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-23071166-3d47-425d-9449-446b9b56554c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:35.787892 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-23071166-3d47-425d-9449-446b9b56554c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:35.788364 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-23071166-3d47-425d-9449-446b9b56554c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:35.866633 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-23071166-3d47-425d-9449-446b9b56554c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:35.875748 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-23071166-3d47-425d-9449-446b9b56554c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.102232 Aug 07 01:37:35.876385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 890/2048] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:35 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:36.910751 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-338e41aa-1f00-4ccf-860f-4b4cb0e1d34f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:36.911360 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-338e41aa-1f00-4ccf-860f-4b4cb0e1d34f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:36.919237 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-338e41aa-1f00-4ccf-860f-4b4cb0e1d34f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:36.919793 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-338e41aa-1f00-4ccf-860f-4b4cb0e1d34f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:36.920319 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-338e41aa-1f00-4ccf-860f-4b4cb0e1d34f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:36.921081 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-338e41aa-1f00-4ccf-860f-4b4cb0e1d34f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:36.921583 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-338e41aa-1f00-4ccf-860f-4b4cb0e1d34f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:36.922056 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-338e41aa-1f00-4ccf-860f-4b4cb0e1d34f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:36.994369 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-338e41aa-1f00-4ccf-860f-4b4cb0e1d34f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:37.009281 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-338e41aa-1f00-4ccf-860f-4b4cb0e1d34f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.101978 Aug 07 01:37:37.010388 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1159/2049] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:36 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:38.030771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bcf4df98-aa99-4224-8b6f-d3c0f0eb42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:38.031446 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bcf4df98-aa99-4224-8b6f-d3c0f0eb42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:38.041895 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bcf4df98-aa99-4224-8b6f-d3c0f0eb42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:38.042762 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bcf4df98-aa99-4224-8b6f-d3c0f0eb42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:38.043545 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bcf4df98-aa99-4224-8b6f-d3c0f0eb42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:38.044895 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bcf4df98-aa99-4224-8b6f-d3c0f0eb42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:38.045679 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bcf4df98-aa99-4224-8b6f-d3c0f0eb42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:38.046449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bcf4df98-aa99-4224-8b6f-d3c0f0eb42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:38.142039 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bcf4df98-aa99-4224-8b6f-d3c0f0eb42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:38.151129 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bcf4df98-aa99-4224-8b6f-d3c0f0eb42bc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.124377 Aug 07 01:37:38.151813 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 891/2050] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:38 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:39.174667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a9d96167-6acd-4419-813a-014bcdc5b175 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:39.175294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a9d96167-6acd-4419-813a-014bcdc5b175 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:39.182243 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9d96167-6acd-4419-813a-014bcdc5b175 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:39.182743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9d96167-6acd-4419-813a-014bcdc5b175 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:39.183213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9d96167-6acd-4419-813a-014bcdc5b175 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:39.183943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9d96167-6acd-4419-813a-014bcdc5b175 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:39.184436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9d96167-6acd-4419-813a-014bcdc5b175 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:39.184864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9d96167-6acd-4419-813a-014bcdc5b175 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:39.253137 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a9d96167-6acd-4419-813a-014bcdc5b175 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:39.261869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a9d96167-6acd-4419-813a-014bcdc5b175 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.091115 Aug 07 01:37:39.262453 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1160/2051] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:39 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:40.286217 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4745492c-b67c-4ac8-97fd-45db9f7d5c45 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:40.287208 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4745492c-b67c-4ac8-97fd-45db9f7d5c45 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:40.295598 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4745492c-b67c-4ac8-97fd-45db9f7d5c45 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:40.296176 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4745492c-b67c-4ac8-97fd-45db9f7d5c45 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:40.296808 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4745492c-b67c-4ac8-97fd-45db9f7d5c45 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:40.297574 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4745492c-b67c-4ac8-97fd-45db9f7d5c45 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:40.298083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4745492c-b67c-4ac8-97fd-45db9f7d5c45 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:40.298634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4745492c-b67c-4ac8-97fd-45db9f7d5c45 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:40.375193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4745492c-b67c-4ac8-97fd-45db9f7d5c45 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:40.383793 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4745492c-b67c-4ac8-97fd-45db9f7d5c45 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.101845 Aug 07 01:37:40.384429 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 892/2052] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:40 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:41.406368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a83e1178-1c51-4d22-8cd7-f41f4ca6ab60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:41.407051 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a83e1178-1c51-4d22-8cd7-f41f4ca6ab60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:41.414215 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a83e1178-1c51-4d22-8cd7-f41f4ca6ab60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:41.414719 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a83e1178-1c51-4d22-8cd7-f41f4ca6ab60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:41.415792 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a83e1178-1c51-4d22-8cd7-f41f4ca6ab60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:41.416553 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a83e1178-1c51-4d22-8cd7-f41f4ca6ab60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:41.417029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a83e1178-1c51-4d22-8cd7-f41f4ca6ab60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:41.417457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a83e1178-1c51-4d22-8cd7-f41f4ca6ab60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:41.488442 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a83e1178-1c51-4d22-8cd7-f41f4ca6ab60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:41.497765 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a83e1178-1c51-4d22-8cd7-f41f4ca6ab60 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.095402 Aug 07 01:37:41.498415 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1161/2053] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:41 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:42.519452 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8d958136-0ca6-4604-a82b-4f7de48ae142 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:42.520135 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8d958136-0ca6-4604-a82b-4f7de48ae142 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:42.528425 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d958136-0ca6-4604-a82b-4f7de48ae142 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:42.529069 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d958136-0ca6-4604-a82b-4f7de48ae142 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:42.529560 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d958136-0ca6-4604-a82b-4f7de48ae142 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:42.530427 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d958136-0ca6-4604-a82b-4f7de48ae142 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:42.530931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d958136-0ca6-4604-a82b-4f7de48ae142 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:42.531473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8d958136-0ca6-4604-a82b-4f7de48ae142 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:42.604006 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8d958136-0ca6-4604-a82b-4f7de48ae142 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:42.612991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8d958136-0ca6-4604-a82b-4f7de48ae142 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.097517 Aug 07 01:37:42.613596 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 893/2054] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:42 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:43.637512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ebf56140-5490-4041-a2bd-7de55dc4415b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:43.638136 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ebf56140-5490-4041-a2bd-7de55dc4415b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:43.645009 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ebf56140-5490-4041-a2bd-7de55dc4415b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:43.645498 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ebf56140-5490-4041-a2bd-7de55dc4415b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:43.645934 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ebf56140-5490-4041-a2bd-7de55dc4415b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:43.646648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ebf56140-5490-4041-a2bd-7de55dc4415b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:43.647101 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ebf56140-5490-4041-a2bd-7de55dc4415b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:43.647524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ebf56140-5490-4041-a2bd-7de55dc4415b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:43.715716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ebf56140-5490-4041-a2bd-7de55dc4415b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:43.724199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ebf56140-5490-4041-a2bd-7de55dc4415b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.091436 Aug 07 01:37:43.724836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1162/2055] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:43 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:44.759158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ab3d603a-fa78-45ee-9985-52a73bcd35e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:44.759946 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ab3d603a-fa78-45ee-9985-52a73bcd35e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:44.766923 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab3d603a-fa78-45ee-9985-52a73bcd35e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:44.767405 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab3d603a-fa78-45ee-9985-52a73bcd35e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:44.767836 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab3d603a-fa78-45ee-9985-52a73bcd35e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:44.768603 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab3d603a-fa78-45ee-9985-52a73bcd35e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:44.769054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab3d603a-fa78-45ee-9985-52a73bcd35e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:44.769477 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab3d603a-fa78-45ee-9985-52a73bcd35e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:44.857362 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ab3d603a-fa78-45ee-9985-52a73bcd35e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:44.871812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ab3d603a-fa78-45ee-9985-52a73bcd35e6 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.116736 Aug 07 01:37:44.873067 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 894/2056] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:44 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:45.894532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a1f7838f-0db1-4b93-aa25-77e0266673a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:45.895193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a1f7838f-0db1-4b93-aa25-77e0266673a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:45.902777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1f7838f-0db1-4b93-aa25-77e0266673a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:45.903270 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1f7838f-0db1-4b93-aa25-77e0266673a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:45.903753 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1f7838f-0db1-4b93-aa25-77e0266673a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:45.904503 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1f7838f-0db1-4b93-aa25-77e0266673a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:45.904960 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1f7838f-0db1-4b93-aa25-77e0266673a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:45.905387 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a1f7838f-0db1-4b93-aa25-77e0266673a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:45.978358 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a1f7838f-0db1-4b93-aa25-77e0266673a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:45.987108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a1f7838f-0db1-4b93-aa25-77e0266673a5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.096405 Aug 07 01:37:45.987711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1163/2057] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:45 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:47.009939 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ffc14bed-1694-447c-b629-40e70540facf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:47.010610 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ffc14bed-1694-447c-b629-40e70540facf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:47.017407 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffc14bed-1694-447c-b629-40e70540facf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:47.017900 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffc14bed-1694-447c-b629-40e70540facf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:47.018335 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffc14bed-1694-447c-b629-40e70540facf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:47.019060 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffc14bed-1694-447c-b629-40e70540facf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:47.019524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffc14bed-1694-447c-b629-40e70540facf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:47.019955 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ffc14bed-1694-447c-b629-40e70540facf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:47.087424 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ffc14bed-1694-447c-b629-40e70540facf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:47.096123 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ffc14bed-1694-447c-b629-40e70540facf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.090616 Aug 07 01:37:47.096772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 895/2058] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:47 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:48.118716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bdf85f62-de8c-41f0-901d-9f635af2eda5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:48.119642 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bdf85f62-de8c-41f0-901d-9f635af2eda5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:48.126375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bdf85f62-de8c-41f0-901d-9f635af2eda5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:48.126872 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bdf85f62-de8c-41f0-901d-9f635af2eda5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:48.127316 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bdf85f62-de8c-41f0-901d-9f635af2eda5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:48.128040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bdf85f62-de8c-41f0-901d-9f635af2eda5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:48.128545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bdf85f62-de8c-41f0-901d-9f635af2eda5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:48.128992 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bdf85f62-de8c-41f0-901d-9f635af2eda5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:48.196832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-bdf85f62-de8c-41f0-901d-9f635af2eda5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:48.205388 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bdf85f62-de8c-41f0-901d-9f635af2eda5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.091356 Aug 07 01:37:48.205982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1164/2059] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:48 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:49.230268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8360fff2-9ac6-469a-b310-0837a0adae20 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:49.231002 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8360fff2-9ac6-469a-b310-0837a0adae20 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:49.233527 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=25 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:37:49.239517 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8360fff2-9ac6-469a-b310-0837a0adae20 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:49.240013 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8360fff2-9ac6-469a-b310-0837a0adae20 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:49.240483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8360fff2-9ac6-469a-b310-0837a0adae20 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:49.241213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8360fff2-9ac6-469a-b310-0837a0adae20 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:49.241673 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8360fff2-9ac6-469a-b310-0837a0adae20 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:49.242103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8360fff2-9ac6-469a-b310-0837a0adae20 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:49.246269 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=69 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:37:49.336791 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8360fff2-9ac6-469a-b310-0837a0adae20 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:49.350975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8360fff2-9ac6-469a-b310-0837a0adae20 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.124527 Aug 07 01:37:49.352854 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 896/2060] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:49 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:50.388065 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7748abc6-e69d-4bba-be3e-076d05b24ed2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:50.388911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7748abc6-e69d-4bba-be3e-076d05b24ed2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:50.395237 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7748abc6-e69d-4bba-be3e-076d05b24ed2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:50.395725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7748abc6-e69d-4bba-be3e-076d05b24ed2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:50.396162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7748abc6-e69d-4bba-be3e-076d05b24ed2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:50.396931 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7748abc6-e69d-4bba-be3e-076d05b24ed2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:50.397385 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7748abc6-e69d-4bba-be3e-076d05b24ed2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:50.397812 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7748abc6-e69d-4bba-be3e-076d05b24ed2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:50.471358 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7748abc6-e69d-4bba-be3e-076d05b24ed2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:50.480458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7748abc6-e69d-4bba-be3e-076d05b24ed2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.096829 Aug 07 01:37:50.481132 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1165/2061] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:50 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:51.503123 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6ad1f35c-a7aa-4e29-a49a-5964827b16f8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:51.503882 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6ad1f35c-a7aa-4e29-a49a-5964827b16f8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:51.511637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6ad1f35c-a7aa-4e29-a49a-5964827b16f8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:51.512180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6ad1f35c-a7aa-4e29-a49a-5964827b16f8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:51.512684 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6ad1f35c-a7aa-4e29-a49a-5964827b16f8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:51.513508 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6ad1f35c-a7aa-4e29-a49a-5964827b16f8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:51.514079 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6ad1f35c-a7aa-4e29-a49a-5964827b16f8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:51.514543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6ad1f35c-a7aa-4e29-a49a-5964827b16f8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:51.587929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6ad1f35c-a7aa-4e29-a49a-5964827b16f8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:51.596501 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6ad1f35c-a7aa-4e29-a49a-5964827b16f8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.097581 Aug 07 01:37:51.597171 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 897/2062] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:51 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:52.618371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8a46b23c-05de-4c74-b25a-450fb4e46350 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:52.619062 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8a46b23c-05de-4c74-b25a-450fb4e46350 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:52.625731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a46b23c-05de-4c74-b25a-450fb4e46350 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:52.626221 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a46b23c-05de-4c74-b25a-450fb4e46350 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:52.626662 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a46b23c-05de-4c74-b25a-450fb4e46350 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:52.627371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a46b23c-05de-4c74-b25a-450fb4e46350 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:52.627825 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a46b23c-05de-4c74-b25a-450fb4e46350 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:52.628274 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8a46b23c-05de-4c74-b25a-450fb4e46350 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:52.696831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8a46b23c-05de-4c74-b25a-450fb4e46350 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:52.708105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8a46b23c-05de-4c74-b25a-450fb4e46350 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.094063 Aug 07 01:37:52.708812 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1166/2063] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:52 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:53.733832 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-697bcaca-d797-4809-9191-e171647d61c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:53.734913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-697bcaca-d797-4809-9191-e171647d61c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:53.745983 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-697bcaca-d797-4809-9191-e171647d61c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:53.746815 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-697bcaca-d797-4809-9191-e171647d61c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:53.747593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-697bcaca-d797-4809-9191-e171647d61c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:53.748830 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-697bcaca-d797-4809-9191-e171647d61c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:53.749591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-697bcaca-d797-4809-9191-e171647d61c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:53.750293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-697bcaca-d797-4809-9191-e171647d61c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:53.826924 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-697bcaca-d797-4809-9191-e171647d61c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:53.836054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-697bcaca-d797-4809-9191-e171647d61c4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.109819 Aug 07 01:37:53.837577 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 898/2064] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:53 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:54.859118 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-351bebf2-8a04-4c2b-80d3-adefca274508 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:54.859878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-351bebf2-8a04-4c2b-80d3-adefca274508 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:54.867607 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-351bebf2-8a04-4c2b-80d3-adefca274508 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:54.868737 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-351bebf2-8a04-4c2b-80d3-adefca274508 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:54.869230 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-351bebf2-8a04-4c2b-80d3-adefca274508 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:54.870016 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-351bebf2-8a04-4c2b-80d3-adefca274508 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:54.870651 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-351bebf2-8a04-4c2b-80d3-adefca274508 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:54.871132 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-351bebf2-8a04-4c2b-80d3-adefca274508 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:54.941584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-351bebf2-8a04-4c2b-80d3-adefca274508 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:54.950464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-351bebf2-8a04-4c2b-80d3-adefca274508 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.095344 Aug 07 01:37:54.951072 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1167/2065] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:54 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:37:55.973866 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-66b4c8ae-4eae-4e09-89c7-152bfe0d6a16 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:55.974587 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-66b4c8ae-4eae-4e09-89c7-152bfe0d6a16 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:55.982500 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-66b4c8ae-4eae-4e09-89c7-152bfe0d6a16 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:55.982986 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-66b4c8ae-4eae-4e09-89c7-152bfe0d6a16 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:55.983418 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-66b4c8ae-4eae-4e09-89c7-152bfe0d6a16 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:55.984131 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-66b4c8ae-4eae-4e09-89c7-152bfe0d6a16 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:55.984633 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-66b4c8ae-4eae-4e09-89c7-152bfe0d6a16 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:55.985059 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-66b4c8ae-4eae-4e09-89c7-152bfe0d6a16 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:56.053531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-66b4c8ae-4eae-4e09-89c7-152bfe0d6a16 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:56.062702 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-66b4c8ae-4eae-4e09-89c7-152bfe0d6a16 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.092760 Aug 07 01:37:56.063352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 899/2066] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:55 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:57.085506 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bf687be5-34ac-49eb-84b2-c779b4c37c4c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:57.086182 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bf687be5-34ac-49eb-84b2-c779b4c37c4c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:57.093643 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bf687be5-34ac-49eb-84b2-c779b4c37c4c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:57.094135 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bf687be5-34ac-49eb-84b2-c779b4c37c4c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:57.094573 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bf687be5-34ac-49eb-84b2-c779b4c37c4c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:57.095344 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bf687be5-34ac-49eb-84b2-c779b4c37c4c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:57.095878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bf687be5-34ac-49eb-84b2-c779b4c37c4c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:57.096341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bf687be5-34ac-49eb-84b2-c779b4c37c4c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:57.168411 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-bf687be5-34ac-49eb-84b2-c779b4c37c4c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:57.177240 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bf687be5-34ac-49eb-84b2-c779b4c37c4c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.095742 Aug 07 01:37:57.177833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1168/2067] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:57 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:58.199092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-65eb51e9-3b1f-4d45-a835-33cb6ed14e82 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:58.199794 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-65eb51e9-3b1f-4d45-a835-33cb6ed14e82 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:58.206802 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65eb51e9-3b1f-4d45-a835-33cb6ed14e82 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:58.207414 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65eb51e9-3b1f-4d45-a835-33cb6ed14e82 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:58.207856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65eb51e9-3b1f-4d45-a835-33cb6ed14e82 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:58.208608 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65eb51e9-3b1f-4d45-a835-33cb6ed14e82 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:58.209061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65eb51e9-3b1f-4d45-a835-33cb6ed14e82 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:58.209487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-65eb51e9-3b1f-4d45-a835-33cb6ed14e82 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:58.285816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-65eb51e9-3b1f-4d45-a835-33cb6ed14e82 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:58.295094 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-65eb51e9-3b1f-4d45-a835-33cb6ed14e82 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.100445 Aug 07 01:37:58.295686 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 900/2068] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:58 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:37:59.317629 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ee6e09c1-b6d4-4156-be2b-3c0791c003ec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:37:59.318317 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ee6e09c1-b6d4-4156-be2b-3c0791c003ec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:37:59.326280 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee6e09c1-b6d4-4156-be2b-3c0791c003ec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:59.326923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee6e09c1-b6d4-4156-be2b-3c0791c003ec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:59.327437 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee6e09c1-b6d4-4156-be2b-3c0791c003ec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:59.328359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee6e09c1-b6d4-4156-be2b-3c0791c003ec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:37:59.328902 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee6e09c1-b6d4-4156-be2b-3c0791c003ec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:37:59.329383 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ee6e09c1-b6d4-4156-be2b-3c0791c003ec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:37:59.406725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ee6e09c1-b6d4-4156-be2b-3c0791c003ec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:37:59.416209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ee6e09c1-b6d4-4156-be2b-3c0791c003ec tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.102496 Aug 07 01:37:59.417022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1169/2069] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:37:59 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:00.443664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-1a7ceb9e-8836-4bc5-a375-4879d735169c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:00.444343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-1a7ceb9e-8836-4bc5-a375-4879d735169c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:00.454242 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a7ceb9e-8836-4bc5-a375-4879d735169c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:00.454803 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a7ceb9e-8836-4bc5-a375-4879d735169c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:00.455468 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a7ceb9e-8836-4bc5-a375-4879d735169c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:00.456579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a7ceb9e-8836-4bc5-a375-4879d735169c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:00.457059 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a7ceb9e-8836-4bc5-a375-4879d735169c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:00.457541 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-1a7ceb9e-8836-4bc5-a375-4879d735169c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:00.530329 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-1a7ceb9e-8836-4bc5-a375-4879d735169c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:00.547313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-1a7ceb9e-8836-4bc5-a375-4879d735169c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.107338 Aug 07 01:38:00.548335 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 901/2070] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:00 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:01.571435 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-aed37bfe-23ed-4327-8c5f-a930a9c3ec48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:01.572069 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-aed37bfe-23ed-4327-8c5f-a930a9c3ec48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:01.578999 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aed37bfe-23ed-4327-8c5f-a930a9c3ec48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:01.579522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aed37bfe-23ed-4327-8c5f-a930a9c3ec48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:01.579982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aed37bfe-23ed-4327-8c5f-a930a9c3ec48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:01.580769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aed37bfe-23ed-4327-8c5f-a930a9c3ec48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:01.581240 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aed37bfe-23ed-4327-8c5f-a930a9c3ec48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:01.581678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-aed37bfe-23ed-4327-8c5f-a930a9c3ec48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:01.587486 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=84 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:38:01.690704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-aed37bfe-23ed-4327-8c5f-a930a9c3ec48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:01.699270 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-aed37bfe-23ed-4327-8c5f-a930a9c3ec48 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.132372 Aug 07 01:38:01.699873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1170/2071] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:01 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:02.734439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b143991e-5b47-4825-9291-c3892e277659 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:02.735090 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b143991e-5b47-4825-9291-c3892e277659 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:02.741912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b143991e-5b47-4825-9291-c3892e277659 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:02.742393 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b143991e-5b47-4825-9291-c3892e277659 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:02.742827 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b143991e-5b47-4825-9291-c3892e277659 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:02.743541 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b143991e-5b47-4825-9291-c3892e277659 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:02.743995 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b143991e-5b47-4825-9291-c3892e277659 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:02.744459 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b143991e-5b47-4825-9291-c3892e277659 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:02.824813 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b143991e-5b47-4825-9291-c3892e277659 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:02.833591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b143991e-5b47-4825-9291-c3892e277659 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.103289 Aug 07 01:38:02.834195 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 902/2072] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:02 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:03.856831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-66b94339-2b5c-4eb8-ae17-c1f2b9e7716e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:03.857520 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-66b94339-2b5c-4eb8-ae17-c1f2b9e7716e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:03.864678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-66b94339-2b5c-4eb8-ae17-c1f2b9e7716e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:03.865239 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-66b94339-2b5c-4eb8-ae17-c1f2b9e7716e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:03.865725 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-66b94339-2b5c-4eb8-ae17-c1f2b9e7716e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:03.866495 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-66b94339-2b5c-4eb8-ae17-c1f2b9e7716e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:03.867078 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-66b94339-2b5c-4eb8-ae17-c1f2b9e7716e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:03.867572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-66b94339-2b5c-4eb8-ae17-c1f2b9e7716e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:03.967525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-66b94339-2b5c-4eb8-ae17-c1f2b9e7716e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:03.976171 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-66b94339-2b5c-4eb8-ae17-c1f2b9e7716e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.123965 Aug 07 01:38:03.976900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1171/2073] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:03 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:04.998235 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9305f844-bae4-4301-8abd-7b50a68518b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:04.999275 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9305f844-bae4-4301-8abd-7b50a68518b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:05.007695 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9305f844-bae4-4301-8abd-7b50a68518b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:05.008881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9305f844-bae4-4301-8abd-7b50a68518b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:05.009529 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9305f844-bae4-4301-8abd-7b50a68518b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:05.010394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9305f844-bae4-4301-8abd-7b50a68518b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:05.010968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9305f844-bae4-4301-8abd-7b50a68518b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:05.011559 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9305f844-bae4-4301-8abd-7b50a68518b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:05.082716 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9305f844-bae4-4301-8abd-7b50a68518b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:05.091877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9305f844-bae4-4301-8abd-7b50a68518b4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.097825 Aug 07 01:38:05.092525 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 903/2074] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:04 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:06.115665 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a5421b6b-de16-48d8-ab58-a36afcf6d41d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:06.116370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a5421b6b-de16-48d8-ab58-a36afcf6d41d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:06.119536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=28 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:38:06.125609 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5421b6b-de16-48d8-ab58-a36afcf6d41d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:06.126158 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5421b6b-de16-48d8-ab58-a36afcf6d41d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:06.126596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5421b6b-de16-48d8-ab58-a36afcf6d41d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:06.127375 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5421b6b-de16-48d8-ab58-a36afcf6d41d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:06.127778 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5421b6b-de16-48d8-ab58-a36afcf6d41d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:06.128211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a5421b6b-de16-48d8-ab58-a36afcf6d41d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:06.201455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a5421b6b-de16-48d8-ab58-a36afcf6d41d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:06.209986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a5421b6b-de16-48d8-ab58-a36afcf6d41d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.099004 Aug 07 01:38:06.210835 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1172/2075] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:06 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:07.233301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-0beddead-29fa-489f-92be-66843d777fdc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:07.234119 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-0beddead-29fa-489f-92be-66843d777fdc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:07.240989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0beddead-29fa-489f-92be-66843d777fdc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:07.241479 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0beddead-29fa-489f-92be-66843d777fdc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:07.241921 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0beddead-29fa-489f-92be-66843d777fdc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:07.242654 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0beddead-29fa-489f-92be-66843d777fdc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:07.243111 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0beddead-29fa-489f-92be-66843d777fdc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:07.243550 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-0beddead-29fa-489f-92be-66843d777fdc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:07.318262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-0beddead-29fa-489f-92be-66843d777fdc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:07.327895 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-0beddead-29fa-489f-92be-66843d777fdc tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.098481 Aug 07 01:38:07.328531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 904/2076] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:07 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:08.350924 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fc87199d-b260-4ec3-8beb-e540f20f9647 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:08.351610 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fc87199d-b260-4ec3-8beb-e540f20f9647 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:08.359124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc87199d-b260-4ec3-8beb-e540f20f9647 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:08.359617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc87199d-b260-4ec3-8beb-e540f20f9647 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:08.360049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc87199d-b260-4ec3-8beb-e540f20f9647 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:08.360799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc87199d-b260-4ec3-8beb-e540f20f9647 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:08.361251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc87199d-b260-4ec3-8beb-e540f20f9647 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:08.361675 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fc87199d-b260-4ec3-8beb-e540f20f9647 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:08.429987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-fc87199d-b260-4ec3-8beb-e540f20f9647 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:08.438777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fc87199d-b260-4ec3-8beb-e540f20f9647 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.091655 Aug 07 01:38:08.439373 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1173/2077] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:08 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:09.459505 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-de6e2e53-eaff-42e4-aa85-8f71f362403c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:09.460165 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-de6e2e53-eaff-42e4-aa85-8f71f362403c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:09.468962 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-de6e2e53-eaff-42e4-aa85-8f71f362403c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:09.469624 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-de6e2e53-eaff-42e4-aa85-8f71f362403c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:09.470181 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-de6e2e53-eaff-42e4-aa85-8f71f362403c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:09.471035 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-de6e2e53-eaff-42e4-aa85-8f71f362403c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:09.471720 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-de6e2e53-eaff-42e4-aa85-8f71f362403c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:09.472278 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-de6e2e53-eaff-42e4-aa85-8f71f362403c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:09.542675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-de6e2e53-eaff-42e4-aa85-8f71f362403c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:09.551476 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-de6e2e53-eaff-42e4-aa85-8f71f362403c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.095982 Aug 07 01:38:09.552087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 905/2078] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:09 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:10.587237 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3da48a2a-b0bc-4a0b-ae38-5f3579826b4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:10.587969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3da48a2a-b0bc-4a0b-ae38-5f3579826b4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:10.595414 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3da48a2a-b0bc-4a0b-ae38-5f3579826b4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:10.595900 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3da48a2a-b0bc-4a0b-ae38-5f3579826b4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:10.596391 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3da48a2a-b0bc-4a0b-ae38-5f3579826b4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:10.597090 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3da48a2a-b0bc-4a0b-ae38-5f3579826b4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:10.597547 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3da48a2a-b0bc-4a0b-ae38-5f3579826b4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:10.597977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3da48a2a-b0bc-4a0b-ae38-5f3579826b4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:10.666504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3da48a2a-b0bc-4a0b-ae38-5f3579826b4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:10.675430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3da48a2a-b0bc-4a0b-ae38-5f3579826b4a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.092785 Aug 07 01:38:10.676031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1174/2079] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:10 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:11.698180 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9ecfc826-f4be-4e3b-97ef-53309cedab3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:11.698799 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9ecfc826-f4be-4e3b-97ef-53309cedab3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:11.708901 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ecfc826-f4be-4e3b-97ef-53309cedab3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:11.709401 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ecfc826-f4be-4e3b-97ef-53309cedab3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:11.709842 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ecfc826-f4be-4e3b-97ef-53309cedab3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:11.710825 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ecfc826-f4be-4e3b-97ef-53309cedab3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:11.712045 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ecfc826-f4be-4e3b-97ef-53309cedab3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:11.712646 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ecfc826-f4be-4e3b-97ef-53309cedab3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:11.788354 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9ecfc826-f4be-4e3b-97ef-53309cedab3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:11.797228 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9ecfc826-f4be-4e3b-97ef-53309cedab3c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.103787 Aug 07 01:38:11.797820 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 906/2080] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:11 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:12.818565 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ab2dc700-0bf2-49fb-8a8a-181c9c6bad75 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:12.819228 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ab2dc700-0bf2-49fb-8a8a-181c9c6bad75 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:12.826584 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab2dc700-0bf2-49fb-8a8a-181c9c6bad75 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:12.827078 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab2dc700-0bf2-49fb-8a8a-181c9c6bad75 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:12.827522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab2dc700-0bf2-49fb-8a8a-181c9c6bad75 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:12.828267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab2dc700-0bf2-49fb-8a8a-181c9c6bad75 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:12.828743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab2dc700-0bf2-49fb-8a8a-181c9c6bad75 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:12.829178 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab2dc700-0bf2-49fb-8a8a-181c9c6bad75 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:12.901626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-ab2dc700-0bf2-49fb-8a8a-181c9c6bad75 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:12.910513 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ab2dc700-0bf2-49fb-8a8a-181c9c6bad75 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.095804 Aug 07 01:38:12.911111 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1175/2081] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:12 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:13.931494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-692b2db8-bd8c-4535-a5da-f997cb092ed5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:13.932143 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-692b2db8-bd8c-4535-a5da-f997cb092ed5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:13.939034 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-692b2db8-bd8c-4535-a5da-f997cb092ed5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:13.939531 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-692b2db8-bd8c-4535-a5da-f997cb092ed5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:13.939973 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-692b2db8-bd8c-4535-a5da-f997cb092ed5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:13.940730 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-692b2db8-bd8c-4535-a5da-f997cb092ed5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:13.941188 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-692b2db8-bd8c-4535-a5da-f997cb092ed5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:13.941622 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-692b2db8-bd8c-4535-a5da-f997cb092ed5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:14.029946 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-692b2db8-bd8c-4535-a5da-f997cb092ed5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:14.038746 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-692b2db8-bd8c-4535-a5da-f997cb092ed5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.111223 Aug 07 01:38:14.039383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 907/2082] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:13 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:15.063515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-fade633d-19f7-4fd7-8aa6-3d75b3cbcbd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:15.064896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-fade633d-19f7-4fd7-8aa6-3d75b3cbcbd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:15.077185 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fade633d-19f7-4fd7-8aa6-3d75b3cbcbd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:15.078076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fade633d-19f7-4fd7-8aa6-3d75b3cbcbd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:15.078846 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fade633d-19f7-4fd7-8aa6-3d75b3cbcbd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:15.080122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fade633d-19f7-4fd7-8aa6-3d75b3cbcbd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:15.080966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fade633d-19f7-4fd7-8aa6-3d75b3cbcbd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:15.081722 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-fade633d-19f7-4fd7-8aa6-3d75b3cbcbd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:15.186032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-fade633d-19f7-4fd7-8aa6-3d75b3cbcbd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:15.197219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-fade633d-19f7-4fd7-8aa6-3d75b3cbcbd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.141325 Aug 07 01:38:15.198074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1176/2083] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:15 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:16.219298 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d54c7b8b-91cd-4bcc-a640-8cb1fd21f15d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:16.219933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d54c7b8b-91cd-4bcc-a640-8cb1fd21f15d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:16.226766 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d54c7b8b-91cd-4bcc-a640-8cb1fd21f15d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:16.227829 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d54c7b8b-91cd-4bcc-a640-8cb1fd21f15d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:16.228312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d54c7b8b-91cd-4bcc-a640-8cb1fd21f15d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:16.229044 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d54c7b8b-91cd-4bcc-a640-8cb1fd21f15d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:16.229504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d54c7b8b-91cd-4bcc-a640-8cb1fd21f15d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:16.229934 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d54c7b8b-91cd-4bcc-a640-8cb1fd21f15d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:16.307126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d54c7b8b-91cd-4bcc-a640-8cb1fd21f15d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:16.316511 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d54c7b8b-91cd-4bcc-a640-8cb1fd21f15d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.101007 Aug 07 01:38:16.317179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 908/2084] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:16 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:17.340047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-26c02db4-9c58-4231-b799-8c567db06e4d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:17.340765 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-26c02db4-9c58-4231-b799-8c567db06e4d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:17.347800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26c02db4-9c58-4231-b799-8c567db06e4d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:17.348321 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26c02db4-9c58-4231-b799-8c567db06e4d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:17.348762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26c02db4-9c58-4231-b799-8c567db06e4d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:17.349487 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26c02db4-9c58-4231-b799-8c567db06e4d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:17.349948 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26c02db4-9c58-4231-b799-8c567db06e4d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:17.350376 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-26c02db4-9c58-4231-b799-8c567db06e4d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:17.418516 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-26c02db4-9c58-4231-b799-8c567db06e4d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:17.427348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-26c02db4-9c58-4231-b799-8c567db06e4d tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.092109 Aug 07 01:38:17.427956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1177/2085] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:17 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:18.450666 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ef14021b-6fee-467e-9e08-bf5d91ec70f3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:18.451429 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ef14021b-6fee-467e-9e08-bf5d91ec70f3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:18.458436 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef14021b-6fee-467e-9e08-bf5d91ec70f3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:18.458932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef14021b-6fee-467e-9e08-bf5d91ec70f3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:18.459374 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef14021b-6fee-467e-9e08-bf5d91ec70f3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:18.460086 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef14021b-6fee-467e-9e08-bf5d91ec70f3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:18.460581 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef14021b-6fee-467e-9e08-bf5d91ec70f3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:18.461305 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ef14021b-6fee-467e-9e08-bf5d91ec70f3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:18.558053 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ef14021b-6fee-467e-9e08-bf5d91ec70f3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:18.566821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ef14021b-6fee-467e-9e08-bf5d91ec70f3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.120679 Aug 07 01:38:18.567429 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 909/2086] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:18 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:19.601653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-76736ae0-2a6d-4682-afbf-f67067e88cb5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:19.602790 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-76736ae0-2a6d-4682-afbf-f67067e88cb5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:19.609381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76736ae0-2a6d-4682-afbf-f67067e88cb5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:19.609875 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76736ae0-2a6d-4682-afbf-f67067e88cb5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:19.610320 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76736ae0-2a6d-4682-afbf-f67067e88cb5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:19.611035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76736ae0-2a6d-4682-afbf-f67067e88cb5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:19.611493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76736ae0-2a6d-4682-afbf-f67067e88cb5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:19.611921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-76736ae0-2a6d-4682-afbf-f67067e88cb5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:19.681796 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-76736ae0-2a6d-4682-afbf-f67067e88cb5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:19.690784 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-76736ae0-2a6d-4682-afbf-f67067e88cb5 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.093101 Aug 07 01:38:19.691392 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1178/2087] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:19 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:20.712383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5476793b-f118-4d29-8bd4-723371926dc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:20.713039 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5476793b-f118-4d29-8bd4-723371926dc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:20.715143 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=28 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:38:20.720946 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5476793b-f118-4d29-8bd4-723371926dc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:20.721620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5476793b-f118-4d29-8bd4-723371926dc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:20.722167 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5476793b-f118-4d29-8bd4-723371926dc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:20.722999 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5476793b-f118-4d29-8bd4-723371926dc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:20.723557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5476793b-f118-4d29-8bd4-723371926dc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:20.724086 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5476793b-f118-4d29-8bd4-723371926dc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:20.727811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=84 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:38:20.792687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5476793b-f118-4d29-8bd4-723371926dc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:20.801461 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5476793b-f118-4d29-8bd4-723371926dc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.093870 Aug 07 01:38:20.802235 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 910/2088] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:20 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:21.824267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-67122a3b-78b7-41d6-b86d-8fddb709a9bf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:21.824933 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-67122a3b-78b7-41d6-b86d-8fddb709a9bf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:21.831844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67122a3b-78b7-41d6-b86d-8fddb709a9bf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:21.832353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67122a3b-78b7-41d6-b86d-8fddb709a9bf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:21.832800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67122a3b-78b7-41d6-b86d-8fddb709a9bf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:21.833524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67122a3b-78b7-41d6-b86d-8fddb709a9bf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:21.833979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67122a3b-78b7-41d6-b86d-8fddb709a9bf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:21.834411 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-67122a3b-78b7-41d6-b86d-8fddb709a9bf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:21.911216 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-67122a3b-78b7-41d6-b86d-8fddb709a9bf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:21.920563 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-67122a3b-78b7-41d6-b86d-8fddb709a9bf tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.100666 Aug 07 01:38:21.921240 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1179/2089] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:21 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:22.956020 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6408a44f-a002-48ab-b695-0bd8f5425fc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:22.956885 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6408a44f-a002-48ab-b695-0bd8f5425fc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:22.964989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6408a44f-a002-48ab-b695-0bd8f5425fc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:22.965567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6408a44f-a002-48ab-b695-0bd8f5425fc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:22.966051 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6408a44f-a002-48ab-b695-0bd8f5425fc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:22.966827 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6408a44f-a002-48ab-b695-0bd8f5425fc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:22.967404 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6408a44f-a002-48ab-b695-0bd8f5425fc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:22.967887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6408a44f-a002-48ab-b695-0bd8f5425fc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:23.037591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6408a44f-a002-48ab-b695-0bd8f5425fc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:23.046399 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6408a44f-a002-48ab-b695-0bd8f5425fc8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.094555 Aug 07 01:38:23.047008 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 911/2090] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:22 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:24.067713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-249cd9f8-cb62-4374-b231-1d067f97d0b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:24.068413 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-249cd9f8-cb62-4374-b231-1d067f97d0b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:24.075367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-249cd9f8-cb62-4374-b231-1d067f97d0b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:24.075867 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-249cd9f8-cb62-4374-b231-1d067f97d0b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:24.076344 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-249cd9f8-cb62-4374-b231-1d067f97d0b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:24.077083 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-249cd9f8-cb62-4374-b231-1d067f97d0b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:24.077547 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-249cd9f8-cb62-4374-b231-1d067f97d0b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:24.077981 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-249cd9f8-cb62-4374-b231-1d067f97d0b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:24.148497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-249cd9f8-cb62-4374-b231-1d067f97d0b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:24.158267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-249cd9f8-cb62-4374-b231-1d067f97d0b3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.094650 Aug 07 01:38:24.159162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1180/2091] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:24 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:25.184197 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-61484d07-3dcc-48d6-a45c-a4ef24ac2379 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:25.184944 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-61484d07-3dcc-48d6-a45c-a4ef24ac2379 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:25.191902 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61484d07-3dcc-48d6-a45c-a4ef24ac2379 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:25.192570 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61484d07-3dcc-48d6-a45c-a4ef24ac2379 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:25.193019 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61484d07-3dcc-48d6-a45c-a4ef24ac2379 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:25.193991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61484d07-3dcc-48d6-a45c-a4ef24ac2379 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:25.194464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61484d07-3dcc-48d6-a45c-a4ef24ac2379 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:25.194992 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61484d07-3dcc-48d6-a45c-a4ef24ac2379 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:25.265894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-61484d07-3dcc-48d6-a45c-a4ef24ac2379 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:25.275656 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-61484d07-3dcc-48d6-a45c-a4ef24ac2379 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.095866 Aug 07 01:38:25.276574 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 912/2092] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:25 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:26.299917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-80c99581-874c-44b2-890d-bcbcb3b3f942 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:26.300875 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-80c99581-874c-44b2-890d-bcbcb3b3f942 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:26.308451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-80c99581-874c-44b2-890d-bcbcb3b3f942 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:26.309030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-80c99581-874c-44b2-890d-bcbcb3b3f942 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:26.309534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-80c99581-874c-44b2-890d-bcbcb3b3f942 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:26.310552 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-80c99581-874c-44b2-890d-bcbcb3b3f942 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:26.311119 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-80c99581-874c-44b2-890d-bcbcb3b3f942 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:26.311739 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-80c99581-874c-44b2-890d-bcbcb3b3f942 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:26.384334 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-80c99581-874c-44b2-890d-bcbcb3b3f942 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:26.392910 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-80c99581-874c-44b2-890d-bcbcb3b3f942 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.097421 Aug 07 01:38:26.393519 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1181/2093] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:26 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:27.414806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-dc718a11-123c-4585-bb5c-c840d1cb206e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:27.415449 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-dc718a11-123c-4585-bb5c-c840d1cb206e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:27.422918 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc718a11-123c-4585-bb5c-c840d1cb206e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:27.423464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc718a11-123c-4585-bb5c-c840d1cb206e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:27.423946 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc718a11-123c-4585-bb5c-c840d1cb206e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:27.424830 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc718a11-123c-4585-bb5c-c840d1cb206e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:27.425330 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc718a11-123c-4585-bb5c-c840d1cb206e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:27.425803 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-dc718a11-123c-4585-bb5c-c840d1cb206e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:27.497640 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-dc718a11-123c-4585-bb5c-c840d1cb206e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:27.506116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-dc718a11-123c-4585-bb5c-c840d1cb206e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.095768 Aug 07 01:38:27.506777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 913/2094] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:27 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:28.035909 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:38:28.036281 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:38:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:38:28.529004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-75ce5724-ebab-4772-9cc8-b788b94d2bd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:28.529639 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-75ce5724-ebab-4772-9cc8-b788b94d2bd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:28.537664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-75ce5724-ebab-4772-9cc8-b788b94d2bd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:28.538255 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-75ce5724-ebab-4772-9cc8-b788b94d2bd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:28.538813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-75ce5724-ebab-4772-9cc8-b788b94d2bd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:28.539604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-75ce5724-ebab-4772-9cc8-b788b94d2bd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:28.540124 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-75ce5724-ebab-4772-9cc8-b788b94d2bd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:28.540635 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-75ce5724-ebab-4772-9cc8-b788b94d2bd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:28.615266 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-75ce5724-ebab-4772-9cc8-b788b94d2bd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:28.631056 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-75ce5724-ebab-4772-9cc8-b788b94d2bd4 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.106807 Aug 07 01:38:28.632121 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1182/2095] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:28 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:29.654728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4c63ec1d-39b7-4114-8c98-26683b7c608e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:29.655399 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4c63ec1d-39b7-4114-8c98-26683b7c608e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:29.662703 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c63ec1d-39b7-4114-8c98-26683b7c608e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:29.663287 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c63ec1d-39b7-4114-8c98-26683b7c608e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:29.663770 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c63ec1d-39b7-4114-8c98-26683b7c608e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:29.665369 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c63ec1d-39b7-4114-8c98-26683b7c608e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:29.665906 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c63ec1d-39b7-4114-8c98-26683b7c608e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:29.666385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4c63ec1d-39b7-4114-8c98-26683b7c608e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:29.735301 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4c63ec1d-39b7-4114-8c98-26683b7c608e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:29.745928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4c63ec1d-39b7-4114-8c98-26683b7c608e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.095146 Aug 07 01:38:29.746535 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 914/2096] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:29 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:30.769294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0d11cc18-69da-45df-9849-82aafa5b38d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:30.769993 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0d11cc18-69da-45df-9849-82aafa5b38d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:30.777428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d11cc18-69da-45df-9849-82aafa5b38d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:30.777922 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d11cc18-69da-45df-9849-82aafa5b38d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:30.778365 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d11cc18-69da-45df-9849-82aafa5b38d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:30.779087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d11cc18-69da-45df-9849-82aafa5b38d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:30.779546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d11cc18-69da-45df-9849-82aafa5b38d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:30.779977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0d11cc18-69da-45df-9849-82aafa5b38d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:30.853593 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-0d11cc18-69da-45df-9849-82aafa5b38d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:30.863663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0d11cc18-69da-45df-9849-82aafa5b38d2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.098779 Aug 07 01:38:30.864305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1183/2097] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:30 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:31.886183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-92e4ab5b-a71f-4fea-9644-e5f617feb229 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:31.886931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-92e4ab5b-a71f-4fea-9644-e5f617feb229 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:31.894090 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-92e4ab5b-a71f-4fea-9644-e5f617feb229 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:31.894734 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-92e4ab5b-a71f-4fea-9644-e5f617feb229 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:31.895227 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-92e4ab5b-a71f-4fea-9644-e5f617feb229 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:31.896004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-92e4ab5b-a71f-4fea-9644-e5f617feb229 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:31.896544 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-92e4ab5b-a71f-4fea-9644-e5f617feb229 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:31.897089 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-92e4ab5b-a71f-4fea-9644-e5f617feb229 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:31.969224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-92e4ab5b-a71f-4fea-9644-e5f617feb229 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:31.982053 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-92e4ab5b-a71f-4fea-9644-e5f617feb229 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.099809 Aug 07 01:38:31.982972 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 915/2098] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:31 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:33.013475 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-15776295-87d8-4202-b4af-96d6923a2405 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:33.014162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-15776295-87d8-4202-b4af-96d6923a2405 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:33.020929 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15776295-87d8-4202-b4af-96d6923a2405 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:33.021455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15776295-87d8-4202-b4af-96d6923a2405 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:33.021908 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15776295-87d8-4202-b4af-96d6923a2405 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:33.022630 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15776295-87d8-4202-b4af-96d6923a2405 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:33.023093 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15776295-87d8-4202-b4af-96d6923a2405 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:33.023529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-15776295-87d8-4202-b4af-96d6923a2405 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:33.027633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=84 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:38:33.096091 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-15776295-87d8-4202-b4af-96d6923a2405 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:33.105166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-15776295-87d8-4202-b4af-96d6923a2405 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.095560 Aug 07 01:38:33.105777 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1184/2099] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:33 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:34.127445 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cb197571-ba60-4bcc-ba5e-32ebd733bf0a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:34.128152 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-cb197571-ba60-4bcc-ba5e-32ebd733bf0a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:34.135722 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cb197571-ba60-4bcc-ba5e-32ebd733bf0a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:34.136209 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cb197571-ba60-4bcc-ba5e-32ebd733bf0a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:34.136676 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cb197571-ba60-4bcc-ba5e-32ebd733bf0a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:34.137395 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cb197571-ba60-4bcc-ba5e-32ebd733bf0a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:34.137846 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cb197571-ba60-4bcc-ba5e-32ebd733bf0a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:34.138268 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cb197571-ba60-4bcc-ba5e-32ebd733bf0a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:34.228664 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-cb197571-ba60-4bcc-ba5e-32ebd733bf0a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:34.234682 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:38:34.235343 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:38:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:38:34.237149 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-cb197571-ba60-4bcc-ba5e-32ebd733bf0a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.113833 Aug 07 01:38:34.237820 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 916/2100] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:34 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:35.264038 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-df4c31aa-fa24-4c42-a7cc-5df52bd7855e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:35.265208 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-df4c31aa-fa24-4c42-a7cc-5df52bd7855e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:35.274358 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df4c31aa-fa24-4c42-a7cc-5df52bd7855e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:35.274863 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df4c31aa-fa24-4c42-a7cc-5df52bd7855e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:35.275310 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df4c31aa-fa24-4c42-a7cc-5df52bd7855e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:35.276044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df4c31aa-fa24-4c42-a7cc-5df52bd7855e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:35.276546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df4c31aa-fa24-4c42-a7cc-5df52bd7855e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:35.276984 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-df4c31aa-fa24-4c42-a7cc-5df52bd7855e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:35.357287 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-df4c31aa-fa24-4c42-a7cc-5df52bd7855e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:35.366484 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-df4c31aa-fa24-4c42-a7cc-5df52bd7855e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.110296 Aug 07 01:38:35.368007 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1185/2101] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:35 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:36.390012 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-16fd066d-2c63-4429-8c02-8639e551b338 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:36.390700 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-16fd066d-2c63-4429-8c02-8639e551b338 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:36.397984 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16fd066d-2c63-4429-8c02-8639e551b338 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:36.398618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16fd066d-2c63-4429-8c02-8639e551b338 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:36.399148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16fd066d-2c63-4429-8c02-8639e551b338 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:36.399913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16fd066d-2c63-4429-8c02-8639e551b338 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:36.400552 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16fd066d-2c63-4429-8c02-8639e551b338 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:36.401023 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-16fd066d-2c63-4429-8c02-8639e551b338 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:36.475089 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-16fd066d-2c63-4429-8c02-8639e551b338 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:36.484135 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-16fd066d-2c63-4429-8c02-8639e551b338 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.098145 Aug 07 01:38:36.484780 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 917/2102] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:36 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:37.505823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e9ac7e56-039e-4f03-bf4e-5d5f4d0799d0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:37.506503 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e9ac7e56-039e-4f03-bf4e-5d5f4d0799d0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:37.509114 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=28 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:38:37.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9ac7e56-039e-4f03-bf4e-5d5f4d0799d0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:37.515511 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9ac7e56-039e-4f03-bf4e-5d5f4d0799d0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:37.515987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9ac7e56-039e-4f03-bf4e-5d5f4d0799d0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:37.516887 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9ac7e56-039e-4f03-bf4e-5d5f4d0799d0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:37.517420 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9ac7e56-039e-4f03-bf4e-5d5f4d0799d0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:37.517881 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e9ac7e56-039e-4f03-bf4e-5d5f4d0799d0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:37.585591 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e9ac7e56-039e-4f03-bf4e-5d5f4d0799d0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:37.594579 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e9ac7e56-039e-4f03-bf4e-5d5f4d0799d0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.092672 Aug 07 01:38:37.595347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1186/2103] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:37 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:38.617040 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fc20c8d6-28d9-43c5-b5af-970041add07a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:38.618167 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fc20c8d6-28d9-43c5-b5af-970041add07a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:38.625420 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fc20c8d6-28d9-43c5-b5af-970041add07a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:38.625912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fc20c8d6-28d9-43c5-b5af-970041add07a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:38.626352 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fc20c8d6-28d9-43c5-b5af-970041add07a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:38.627082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fc20c8d6-28d9-43c5-b5af-970041add07a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:38.627543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fc20c8d6-28d9-43c5-b5af-970041add07a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:38.627974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fc20c8d6-28d9-43c5-b5af-970041add07a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:38.698620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fc20c8d6-28d9-43c5-b5af-970041add07a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:38.713780 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fc20c8d6-28d9-43c5-b5af-970041add07a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.100230 Aug 07 01:38:38.714812 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 918/2104] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:38 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:39.752455 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8cdf91e5-068a-4202-9750-7101bda24396 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:39.753521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8cdf91e5-068a-4202-9750-7101bda24396 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:39.765352 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8cdf91e5-068a-4202-9750-7101bda24396 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:39.766223 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8cdf91e5-068a-4202-9750-7101bda24396 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:39.766995 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8cdf91e5-068a-4202-9750-7101bda24396 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:39.768299 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8cdf91e5-068a-4202-9750-7101bda24396 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:39.769129 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8cdf91e5-068a-4202-9750-7101bda24396 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:39.769903 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8cdf91e5-068a-4202-9750-7101bda24396 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:39.846878 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8cdf91e5-068a-4202-9750-7101bda24396 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:39.860369 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8cdf91e5-068a-4202-9750-7101bda24396 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.115078 Aug 07 01:38:39.861285 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1187/2105] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:39 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:40.894307 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9e1516e3-32bf-4a68-bca0-8c1ccb284422 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:40.895128 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9e1516e3-32bf-4a68-bca0-8c1ccb284422 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:40.902371 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9e1516e3-32bf-4a68-bca0-8c1ccb284422 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:40.902866 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9e1516e3-32bf-4a68-bca0-8c1ccb284422 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:40.903305 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9e1516e3-32bf-4a68-bca0-8c1ccb284422 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:40.904038 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9e1516e3-32bf-4a68-bca0-8c1ccb284422 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:40.904536 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9e1516e3-32bf-4a68-bca0-8c1ccb284422 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:40.904972 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9e1516e3-32bf-4a68-bca0-8c1ccb284422 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:40.973593 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9e1516e3-32bf-4a68-bca0-8c1ccb284422 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:40.982622 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9e1516e3-32bf-4a68-bca0-8c1ccb284422 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.092467 Aug 07 01:38:40.983228 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 919/2106] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:40 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:42.006801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3ae47828-cf0b-422d-9e1b-6f23962fe71f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:42.008027 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3ae47828-cf0b-422d-9e1b-6f23962fe71f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:42.015472 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae47828-cf0b-422d-9e1b-6f23962fe71f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:42.016020 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae47828-cf0b-422d-9e1b-6f23962fe71f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:42.016576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae47828-cf0b-422d-9e1b-6f23962fe71f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:42.017344 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae47828-cf0b-422d-9e1b-6f23962fe71f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:42.017859 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae47828-cf0b-422d-9e1b-6f23962fe71f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:42.018452 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3ae47828-cf0b-422d-9e1b-6f23962fe71f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:42.115450 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3ae47828-cf0b-422d-9e1b-6f23962fe71f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:42.125157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3ae47828-cf0b-422d-9e1b-6f23962fe71f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.124215 Aug 07 01:38:42.125762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1188/2107] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:41 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:43.148425 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2337ee7f-8a64-446d-84c6-4861ef1bb6a3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:43.149108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2337ee7f-8a64-446d-84c6-4861ef1bb6a3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:43.156620 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2337ee7f-8a64-446d-84c6-4861ef1bb6a3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:43.157117 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2337ee7f-8a64-446d-84c6-4861ef1bb6a3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:43.157566 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2337ee7f-8a64-446d-84c6-4861ef1bb6a3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:43.158389 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2337ee7f-8a64-446d-84c6-4861ef1bb6a3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:43.158787 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2337ee7f-8a64-446d-84c6-4861ef1bb6a3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:43.159222 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2337ee7f-8a64-446d-84c6-4861ef1bb6a3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:43.226675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2337ee7f-8a64-446d-84c6-4861ef1bb6a3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:43.235205 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2337ee7f-8a64-446d-84c6-4861ef1bb6a3 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.090754 Aug 07 01:38:43.235809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 920/2108] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:43 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:44.260504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bb7f0fd5-525b-45a0-898e-9f408bc80ee0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:44.261585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bb7f0fd5-525b-45a0-898e-9f408bc80ee0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:44.273386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb7f0fd5-525b-45a0-898e-9f408bc80ee0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:44.274251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb7f0fd5-525b-45a0-898e-9f408bc80ee0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:44.274997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb7f0fd5-525b-45a0-898e-9f408bc80ee0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:44.276731 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb7f0fd5-525b-45a0-898e-9f408bc80ee0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:44.278021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb7f0fd5-525b-45a0-898e-9f408bc80ee0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:44.278021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bb7f0fd5-525b-45a0-898e-9f408bc80ee0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:44.374762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-bb7f0fd5-525b-45a0-898e-9f408bc80ee0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:44.383925 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bb7f0fd5-525b-45a0-898e-9f408bc80ee0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.130301 Aug 07 01:38:44.384735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1189/2109] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:44 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:44.549195 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:38:44.549523 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:38:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:38:45.405866 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5d0ce1c1-42ee-41a4-b43a-7292e09d87f9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:45.406572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5d0ce1c1-42ee-41a4-b43a-7292e09d87f9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:45.414012 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d0ce1c1-42ee-41a4-b43a-7292e09d87f9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:45.414565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d0ce1c1-42ee-41a4-b43a-7292e09d87f9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:45.415058 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d0ce1c1-42ee-41a4-b43a-7292e09d87f9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:45.415861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d0ce1c1-42ee-41a4-b43a-7292e09d87f9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:45.416439 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d0ce1c1-42ee-41a4-b43a-7292e09d87f9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:45.417016 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-5d0ce1c1-42ee-41a4-b43a-7292e09d87f9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:45.498586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5d0ce1c1-42ee-41a4-b43a-7292e09d87f9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:45.507450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5d0ce1c1-42ee-41a4-b43a-7292e09d87f9 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.105573 Aug 07 01:38:45.508733 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 921/2110] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:45 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:46.529978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-25f5bbe9-92e0-4279-ad3b-537294ad953c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:46.530744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-25f5bbe9-92e0-4279-ad3b-537294ad953c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:46.538667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25f5bbe9-92e0-4279-ad3b-537294ad953c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:46.539713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25f5bbe9-92e0-4279-ad3b-537294ad953c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:46.540200 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25f5bbe9-92e0-4279-ad3b-537294ad953c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:46.541025 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25f5bbe9-92e0-4279-ad3b-537294ad953c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:46.541538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25f5bbe9-92e0-4279-ad3b-537294ad953c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:46.542080 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-25f5bbe9-92e0-4279-ad3b-537294ad953c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:46.612087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-25f5bbe9-92e0-4279-ad3b-537294ad953c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:46.621483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-25f5bbe9-92e0-4279-ad3b-537294ad953c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.095976 Aug 07 01:38:46.622073 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1190/2111] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:46 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:47.642829 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-23649acb-b6c6-41b2-9fc9-5812849db59a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:47.643727 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-23649acb-b6c6-41b2-9fc9-5812849db59a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:47.650960 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-23649acb-b6c6-41b2-9fc9-5812849db59a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:47.651459 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-23649acb-b6c6-41b2-9fc9-5812849db59a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:47.651901 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-23649acb-b6c6-41b2-9fc9-5812849db59a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:47.652662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-23649acb-b6c6-41b2-9fc9-5812849db59a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:47.653122 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-23649acb-b6c6-41b2-9fc9-5812849db59a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:47.653555 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-23649acb-b6c6-41b2-9fc9-5812849db59a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:47.727849 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-23649acb-b6c6-41b2-9fc9-5812849db59a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:47.742915 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-23649acb-b6c6-41b2-9fc9-5812849db59a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.103709 Aug 07 01:38:47.744020 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 922/2112] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:47 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:48.778061 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-54593dde-02b4-4020-b212-436cd9d4439e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:48.779043 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-54593dde-02b4-4020-b212-436cd9d4439e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:48.786712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54593dde-02b4-4020-b212-436cd9d4439e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:48.787206 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54593dde-02b4-4020-b212-436cd9d4439e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:48.787646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54593dde-02b4-4020-b212-436cd9d4439e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:48.788394 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54593dde-02b4-4020-b212-436cd9d4439e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:48.789458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54593dde-02b4-4020-b212-436cd9d4439e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:48.790192 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54593dde-02b4-4020-b212-436cd9d4439e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:48.864614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-54593dde-02b4-4020-b212-436cd9d4439e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:48.874271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-54593dde-02b4-4020-b212-436cd9d4439e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.100573 Aug 07 01:38:48.874885 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1191/2113] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:48 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:49.894754 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f8ad17f0-2847-41fe-8175-5820d070cc01 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:49.895473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f8ad17f0-2847-41fe-8175-5820d070cc01 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:49.903292 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8ad17f0-2847-41fe-8175-5820d070cc01 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:49.903788 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8ad17f0-2847-41fe-8175-5820d070cc01 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:49.904229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8ad17f0-2847-41fe-8175-5820d070cc01 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:49.904994 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8ad17f0-2847-41fe-8175-5820d070cc01 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:49.905453 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8ad17f0-2847-41fe-8175-5820d070cc01 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:49.905881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f8ad17f0-2847-41fe-8175-5820d070cc01 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:49.986007 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f8ad17f0-2847-41fe-8175-5820d070cc01 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:49.995048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f8ad17f0-2847-41fe-8175-5820d070cc01 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.104123 Aug 07 01:38:49.995645 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 923/2114] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:49 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:51.017540 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-685f701d-ba3f-475a-b436-e4d9e162684e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:51.018278 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-685f701d-ba3f-475a-b436-e4d9e162684e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:51.024952 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-685f701d-ba3f-475a-b436-e4d9e162684e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:51.025494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-685f701d-ba3f-475a-b436-e4d9e162684e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:51.025989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-685f701d-ba3f-475a-b436-e4d9e162684e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:51.026822 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-685f701d-ba3f-475a-b436-e4d9e162684e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:51.027387 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-685f701d-ba3f-475a-b436-e4d9e162684e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:51.027917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-685f701d-ba3f-475a-b436-e4d9e162684e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:51.095817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-685f701d-ba3f-475a-b436-e4d9e162684e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:51.105098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-685f701d-ba3f-475a-b436-e4d9e162684e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.091569 Aug 07 01:38:51.105688 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1192/2115] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:51 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:52.126554 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-772f3c74-1c0e-44d1-9428-9facc39fcaeb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:52.127209 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-772f3c74-1c0e-44d1-9428-9facc39fcaeb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:52.129696 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=28 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:38:52.136426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-772f3c74-1c0e-44d1-9428-9facc39fcaeb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:52.136926 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-772f3c74-1c0e-44d1-9428-9facc39fcaeb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:52.137372 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-772f3c74-1c0e-44d1-9428-9facc39fcaeb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:52.138127 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-772f3c74-1c0e-44d1-9428-9facc39fcaeb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:52.138588 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-772f3c74-1c0e-44d1-9428-9facc39fcaeb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:52.139024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-772f3c74-1c0e-44d1-9428-9facc39fcaeb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:52.142275 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=84 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:38:52.208509 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-772f3c74-1c0e-44d1-9428-9facc39fcaeb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:52.219070 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-772f3c74-1c0e-44d1-9428-9facc39fcaeb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.096055 Aug 07 01:38:52.220149 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 924/2116] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:52 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:38:53.242632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-345cacfd-8f12-4da0-9a0c-edf21dbac39e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:53.243356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-345cacfd-8f12-4da0-9a0c-edf21dbac39e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:53.250050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-345cacfd-8f12-4da0-9a0c-edf21dbac39e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:53.250539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-345cacfd-8f12-4da0-9a0c-edf21dbac39e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:53.250972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-345cacfd-8f12-4da0-9a0c-edf21dbac39e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:53.251690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-345cacfd-8f12-4da0-9a0c-edf21dbac39e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:53.252131 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-345cacfd-8f12-4da0-9a0c-edf21dbac39e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:53.252603 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-345cacfd-8f12-4da0-9a0c-edf21dbac39e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:53.323695 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-345cacfd-8f12-4da0-9a0c-edf21dbac39e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:53.332000 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-345cacfd-8f12-4da0-9a0c-edf21dbac39e tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.093724 Aug 07 01:38:53.332613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1193/2117] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:53 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:54.353154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-57db0e72-484f-423c-88a1-d80eab322881 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:54.353837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-57db0e72-484f-423c-88a1-d80eab322881 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:54.360641 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57db0e72-484f-423c-88a1-d80eab322881 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:54.361137 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57db0e72-484f-423c-88a1-d80eab322881 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:54.361581 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57db0e72-484f-423c-88a1-d80eab322881 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:54.362306 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57db0e72-484f-423c-88a1-d80eab322881 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:54.362764 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57db0e72-484f-423c-88a1-d80eab322881 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:54.363200 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-57db0e72-484f-423c-88a1-d80eab322881 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:54.448224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-57db0e72-484f-423c-88a1-d80eab322881 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:54.457432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-57db0e72-484f-423c-88a1-d80eab322881 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.108144 Aug 07 01:38:54.458036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 925/2118] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:54 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:55.480762 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-78508b55-8926-4857-bde2-f01b79cf998a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:55.481482 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-78508b55-8926-4857-bde2-f01b79cf998a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:55.488854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78508b55-8926-4857-bde2-f01b79cf998a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:55.489347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78508b55-8926-4857-bde2-f01b79cf998a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:55.489788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78508b55-8926-4857-bde2-f01b79cf998a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:55.490500 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78508b55-8926-4857-bde2-f01b79cf998a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:55.490983 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78508b55-8926-4857-bde2-f01b79cf998a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:55.491406 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78508b55-8926-4857-bde2-f01b79cf998a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:55.562186 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-78508b55-8926-4857-bde2-f01b79cf998a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:55.575739 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-78508b55-8926-4857-bde2-f01b79cf998a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.098939 Aug 07 01:38:55.576812 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1194/2119] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:55 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:56.609257 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-2a468991-cc99-476d-901f-d8fe217e712b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:56.609916 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-2a468991-cc99-476d-901f-d8fe217e712b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:56.617388 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a468991-cc99-476d-901f-d8fe217e712b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:56.617892 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a468991-cc99-476d-901f-d8fe217e712b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:56.618332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a468991-cc99-476d-901f-d8fe217e712b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:56.619050 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a468991-cc99-476d-901f-d8fe217e712b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:56.619507 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a468991-cc99-476d-901f-d8fe217e712b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:56.619933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-2a468991-cc99-476d-901f-d8fe217e712b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:56.685956 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-2a468991-cc99-476d-901f-d8fe217e712b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:56.694356 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-2a468991-cc99-476d-901f-d8fe217e712b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.088911 Aug 07 01:38:56.694956 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 926/2120] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:56 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:57.716367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-69688e58-9b48-4da6-9331-c3937da164d8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:57.717021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-69688e58-9b48-4da6-9331-c3937da164d8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:57.723641 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69688e58-9b48-4da6-9331-c3937da164d8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:57.724180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69688e58-9b48-4da6-9331-c3937da164d8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:57.724701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69688e58-9b48-4da6-9331-c3937da164d8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:57.725459 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69688e58-9b48-4da6-9331-c3937da164d8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:57.725946 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69688e58-9b48-4da6-9331-c3937da164d8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:57.726408 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-69688e58-9b48-4da6-9331-c3937da164d8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:57.796348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-69688e58-9b48-4da6-9331-c3937da164d8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:57.811054 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-69688e58-9b48-4da6-9331-c3937da164d8 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.098373 Aug 07 01:38:57.812108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1195/2121] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:57 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:58.847315 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-df074bc2-0d96-4e14-a46d-0b0f7a766970 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:58.847962 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-df074bc2-0d96-4e14-a46d-0b0f7a766970 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:58.855894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-df074bc2-0d96-4e14-a46d-0b0f7a766970 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:58.856497 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-df074bc2-0d96-4e14-a46d-0b0f7a766970 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:58.856979 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-df074bc2-0d96-4e14-a46d-0b0f7a766970 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:58.857744 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-df074bc2-0d96-4e14-a46d-0b0f7a766970 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:58.858295 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-df074bc2-0d96-4e14-a46d-0b0f7a766970 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:58.858785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-df074bc2-0d96-4e14-a46d-0b0f7a766970 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:58.927564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-df074bc2-0d96-4e14-a46d-0b0f7a766970 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:38:58.936965 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-df074bc2-0d96-4e14-a46d-0b0f7a766970 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.093480 Aug 07 01:38:58.937558 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 927/2122] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:58 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:38:59.958712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e0ee2288-8981-4243-8396-2873ed446e0b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:38:59.959734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e0ee2288-8981-4243-8396-2873ed446e0b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:38:59.979538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ee2288-8981-4243-8396-2873ed446e0b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:59.980460 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ee2288-8981-4243-8396-2873ed446e0b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:59.981179 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ee2288-8981-4243-8396-2873ed446e0b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:38:59.982359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ee2288-8981-4243-8396-2873ed446e0b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:38:59.983151 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ee2288-8981-4243-8396-2873ed446e0b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:38:59.983854 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e0ee2288-8981-4243-8396-2873ed446e0b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:00.082239 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e0ee2288-8981-4243-8396-2873ed446e0b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:00.091159 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e0ee2288-8981-4243-8396-2873ed446e0b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.136831 Aug 07 01:39:00.091774 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1196/2123] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:38:59 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:39:01.113422 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d446aaf0-af6e-4e05-bdf9-7836ac543b4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:01.114103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d446aaf0-af6e-4e05-bdf9-7836ac543b4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:01.122004 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d446aaf0-af6e-4e05-bdf9-7836ac543b4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:01.122586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d446aaf0-af6e-4e05-bdf9-7836ac543b4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:01.123069 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d446aaf0-af6e-4e05-bdf9-7836ac543b4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:01.123840 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d446aaf0-af6e-4e05-bdf9-7836ac543b4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:01.124405 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d446aaf0-af6e-4e05-bdf9-7836ac543b4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:01.124889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d446aaf0-af6e-4e05-bdf9-7836ac543b4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:01.194052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d446aaf0-af6e-4e05-bdf9-7836ac543b4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:01.208982 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d446aaf0-af6e-4e05-bdf9-7836ac543b4f tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.099213 Aug 07 01:39:01.210009 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 928/2124] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:01 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:39:02.241377 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6ee6e8b2-20e1-46c4-b610-3af2fe301e28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:02.242674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6ee6e8b2-20e1-46c4-b610-3af2fe301e28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:02.249723 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ee6e8b2-20e1-46c4-b610-3af2fe301e28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:02.250299 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ee6e8b2-20e1-46c4-b610-3af2fe301e28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:02.250780 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ee6e8b2-20e1-46c4-b610-3af2fe301e28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:02.251613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ee6e8b2-20e1-46c4-b610-3af2fe301e28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:02.252108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ee6e8b2-20e1-46c4-b610-3af2fe301e28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:02.252668 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6ee6e8b2-20e1-46c4-b610-3af2fe301e28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:02.327010 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6ee6e8b2-20e1-46c4-b610-3af2fe301e28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:02.336118 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6ee6e8b2-20e1-46c4-b610-3af2fe301e28 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.098691 Aug 07 01:39:02.336987 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1197/2125] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:02 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:39:03.360010 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b83bd8c1-5b0f-442c-b49e-b7f4379593a2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:03.361178 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b83bd8c1-5b0f-442c-b49e-b7f4379593a2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:03.372820 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b83bd8c1-5b0f-442c-b49e-b7f4379593a2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:03.373486 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b83bd8c1-5b0f-442c-b49e-b7f4379593a2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:03.374029 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b83bd8c1-5b0f-442c-b49e-b7f4379593a2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:03.375290 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b83bd8c1-5b0f-442c-b49e-b7f4379593a2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:03.376140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b83bd8c1-5b0f-442c-b49e-b7f4379593a2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:03.376992 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b83bd8c1-5b0f-442c-b49e-b7f4379593a2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:03.459570 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b83bd8c1-5b0f-442c-b49e-b7f4379593a2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:03.474057 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b83bd8c1-5b0f-442c-b49e-b7f4379593a2 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.119454 Aug 07 01:39:03.475123 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 929/2126] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:03 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:39:04.507945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-09c18134-b8d0-4132-9269-1b7b1477f999 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:04.508949 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-09c18134-b8d0-4132-9269-1b7b1477f999 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:04.515979 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09c18134-b8d0-4132-9269-1b7b1477f999 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:04.516754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09c18134-b8d0-4132-9269-1b7b1477f999 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:04.517414 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09c18134-b8d0-4132-9269-1b7b1477f999 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:04.518256 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09c18134-b8d0-4132-9269-1b7b1477f999 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:04.518923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09c18134-b8d0-4132-9269-1b7b1477f999 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:04.519342 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-09c18134-b8d0-4132-9269-1b7b1477f999 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:04.522795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=84 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:39:04.586953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-09c18134-b8d0-4132-9269-1b7b1477f999 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:04.595906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-09c18134-b8d0-4132-9269-1b7b1477f999 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.092481 Aug 07 01:39:04.596542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1198/2127] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:04 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:39:05.755240 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f075eefb-12df-4849-b9b4-2de11eb03efb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:05.755889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f075eefb-12df-4849-b9b4-2de11eb03efb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:05.763183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f075eefb-12df-4849-b9b4-2de11eb03efb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:05.763661 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f075eefb-12df-4849-b9b4-2de11eb03efb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:05.764106 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f075eefb-12df-4849-b9b4-2de11eb03efb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:05.764863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f075eefb-12df-4849-b9b4-2de11eb03efb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:05.765324 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f075eefb-12df-4849-b9b4-2de11eb03efb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:05.765754 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f075eefb-12df-4849-b9b4-2de11eb03efb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:05.833081 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f075eefb-12df-4849-b9b4-2de11eb03efb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:05.843753 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f075eefb-12df-4849-b9b4-2de11eb03efb tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.230304 Aug 07 01:39:05.844397 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 930/2128] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:05 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:39:06.866094 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f4d65b83-51de-4aac-bc02-af5fd979fe95 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:06.867364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f4d65b83-51de-4aac-bc02-af5fd979fe95 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:06.874917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4d65b83-51de-4aac-bc02-af5fd979fe95 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:06.875488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4d65b83-51de-4aac-bc02-af5fd979fe95 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:06.875997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4d65b83-51de-4aac-bc02-af5fd979fe95 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:06.876810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4d65b83-51de-4aac-bc02-af5fd979fe95 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:06.877282 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4d65b83-51de-4aac-bc02-af5fd979fe95 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:06.877721 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4d65b83-51de-4aac-bc02-af5fd979fe95 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:06.949595 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f4d65b83-51de-4aac-bc02-af5fd979fe95 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:06.959428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f4d65b83-51de-4aac-bc02-af5fd979fe95 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.097663 Aug 07 01:39:06.960121 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1199/2129] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:06 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:39:07.988628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-4a60eaa9-0903-445d-bf79-45d53c40c954 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:07.989302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-4a60eaa9-0903-445d-bf79-45d53c40c954 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:07.997082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a60eaa9-0903-445d-bf79-45d53c40c954 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:07.997818 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a60eaa9-0903-445d-bf79-45d53c40c954 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:07.998366 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a60eaa9-0903-445d-bf79-45d53c40c954 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:07.999219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a60eaa9-0903-445d-bf79-45d53c40c954 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:07.999777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a60eaa9-0903-445d-bf79-45d53c40c954 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:08.000360 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-4a60eaa9-0903-445d-bf79-45d53c40c954 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:08.040256 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:39:08.040955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:39:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:39:08.070307 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-4a60eaa9-0903-445d-bf79-45d53c40c954 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:08.079176 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-4a60eaa9-0903-445d-bf79-45d53c40c954 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.094745 Aug 07 01:39:08.079778 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 931/2130] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:07 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:39:09.101600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-defa9401-eaa6-4619-95f6-1d7cdc3bc3f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:09.102629 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-defa9401-eaa6-4619-95f6-1d7cdc3bc3f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:09.105381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=28 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:39:09.111500 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-defa9401-eaa6-4619-95f6-1d7cdc3bc3f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:09.112264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-defa9401-eaa6-4619-95f6-1d7cdc3bc3f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:09.112898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-defa9401-eaa6-4619-95f6-1d7cdc3bc3f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:09.113760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-defa9401-eaa6-4619-95f6-1d7cdc3bc3f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:09.114546 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-defa9401-eaa6-4619-95f6-1d7cdc3bc3f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:09.115294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-defa9401-eaa6-4619-95f6-1d7cdc3bc3f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:09.186677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-defa9401-eaa6-4619-95f6-1d7cdc3bc3f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:09.195412 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-defa9401-eaa6-4619-95f6-1d7cdc3bc3f0 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.098200 Aug 07 01:39:09.196002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1200/2131] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:09 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:39:10.216562 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-24f54162-e292-4cf9-8df9-f65499b49427 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:10.217213 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-24f54162-e292-4cf9-8df9-f65499b49427 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:10.224176 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-24f54162-e292-4cf9-8df9-f65499b49427 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:10.224711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-24f54162-e292-4cf9-8df9-f65499b49427 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:10.225157 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-24f54162-e292-4cf9-8df9-f65499b49427 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:10.225893 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-24f54162-e292-4cf9-8df9-f65499b49427 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:10.226363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-24f54162-e292-4cf9-8df9-f65499b49427 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:10.227271 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-24f54162-e292-4cf9-8df9-f65499b49427 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:10.299464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-24f54162-e292-4cf9-8df9-f65499b49427 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:10.309689 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-24f54162-e292-4cf9-8df9-f65499b49427 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.097566 Aug 07 01:39:10.310278 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 932/2132] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:10 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:39:11.335059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-32a4f201-c834-4568-ac4d-4f2ff911f88c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:11.335772 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-32a4f201-c834-4568-ac4d-4f2ff911f88c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:11.344209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32a4f201-c834-4568-ac4d-4f2ff911f88c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:11.344853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32a4f201-c834-4568-ac4d-4f2ff911f88c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:11.345346 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32a4f201-c834-4568-ac4d-4f2ff911f88c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:11.346111 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32a4f201-c834-4568-ac4d-4f2ff911f88c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:11.346626 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32a4f201-c834-4568-ac4d-4f2ff911f88c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:11.347103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-32a4f201-c834-4568-ac4d-4f2ff911f88c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:11.416316 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-32a4f201-c834-4568-ac4d-4f2ff911f88c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:11.425561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-32a4f201-c834-4568-ac4d-4f2ff911f88c tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.098213 Aug 07 01:39:11.426326 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1201/2133] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:11 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:39:12.447624 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c6d76698-0937-48f7-9900-e6460ffd2327 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:12.448321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c6d76698-0937-48f7-9900-e6460ffd2327 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:12.456403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6d76698-0937-48f7-9900-e6460ffd2327 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:12.456972 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6d76698-0937-48f7-9900-e6460ffd2327 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:12.457541 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6d76698-0937-48f7-9900-e6460ffd2327 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:12.458391 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6d76698-0937-48f7-9900-e6460ffd2327 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:12.458971 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6d76698-0937-48f7-9900-e6460ffd2327 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:12.459600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c6d76698-0937-48f7-9900-e6460ffd2327 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:12.536553 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c6d76698-0937-48f7-9900-e6460ffd2327 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:12.552577 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c6d76698-0937-48f7-9900-e6460ffd2327 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.108696 Aug 07 01:39:12.553735 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 933/2134] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:12 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:39:13.589961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-892227c2-7330-42dd-a40a-924bfe3667da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:13.590700 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-892227c2-7330-42dd-a40a-924bfe3667da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:13.598103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-892227c2-7330-42dd-a40a-924bfe3667da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:13.598580 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-892227c2-7330-42dd-a40a-924bfe3667da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:13.599028 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-892227c2-7330-42dd-a40a-924bfe3667da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:13.599788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-892227c2-7330-42dd-a40a-924bfe3667da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:13.600257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-892227c2-7330-42dd-a40a-924bfe3667da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:13.600708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-892227c2-7330-42dd-a40a-924bfe3667da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:13.671072 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-892227c2-7330-42dd-a40a-924bfe3667da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:13.680381 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-892227c2-7330-42dd-a40a-924bfe3667da tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.094841 Aug 07 01:39:13.680988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1202/2135] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:13 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:39:14.716270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8bde517a-372b-46e9-8335-a716521a9d3a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:14.716934 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8bde517a-372b-46e9-8335-a716521a9d3a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:14.723824 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8bde517a-372b-46e9-8335-a716521a9d3a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:14.724348 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8bde517a-372b-46e9-8335-a716521a9d3a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:14.724804 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8bde517a-372b-46e9-8335-a716521a9d3a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:14.725532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8bde517a-372b-46e9-8335-a716521a9d3a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:14.725989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8bde517a-372b-46e9-8335-a716521a9d3a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:14.726419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8bde517a-372b-46e9-8335-a716521a9d3a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:14.793964 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8bde517a-372b-46e9-8335-a716521a9d3a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:14.802401 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8bde517a-372b-46e9-8335-a716521a9d3a tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.091068 Aug 07 01:39:14.802994 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 934/2136] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:14 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:39:15.829509 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-268eb51d-c565-41fb-bb25-9e6e5d52a17b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:15.830678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-268eb51d-c565-41fb-bb25-9e6e5d52a17b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:15.844109 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-268eb51d-c565-41fb-bb25-9e6e5d52a17b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:15.845040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-268eb51d-c565-41fb-bb25-9e6e5d52a17b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:15.845921 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-268eb51d-c565-41fb-bb25-9e6e5d52a17b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:15.847226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-268eb51d-c565-41fb-bb25-9e6e5d52a17b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:15.848096 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-268eb51d-c565-41fb-bb25-9e6e5d52a17b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:15.848870 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-268eb51d-c565-41fb-bb25-9e6e5d52a17b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:15.963955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-268eb51d-c565-41fb-bb25-9e6e5d52a17b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:15.973431 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-268eb51d-c565-41fb-bb25-9e6e5d52a17b tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.151600 Aug 07 01:39:15.974614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1203/2137] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:15 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:39:16.997606 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-944c4b2e-0353-4f3c-9c02-212a2e2c75df tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:16.998275 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-944c4b2e-0353-4f3c-9c02-212a2e2c75df tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:17.005293 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-944c4b2e-0353-4f3c-9c02-212a2e2c75df tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:17.005783 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-944c4b2e-0353-4f3c-9c02-212a2e2c75df tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:17.006218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-944c4b2e-0353-4f3c-9c02-212a2e2c75df tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:17.006937 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-944c4b2e-0353-4f3c-9c02-212a2e2c75df tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:17.007881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-944c4b2e-0353-4f3c-9c02-212a2e2c75df tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:17.008339 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-944c4b2e-0353-4f3c-9c02-212a2e2c75df tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:17.078962 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-944c4b2e-0353-4f3c-9c02-212a2e2c75df tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:17.088935 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-944c4b2e-0353-4f3c-9c02-212a2e2c75df tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.095133 Aug 07 01:39:17.089543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 935/2138] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:16 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:39:18.110467 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-78b780b8-88a8-40e8-a55e-cc60ed214515 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:18.111115 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-78b780b8-88a8-40e8-a55e-cc60ed214515 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:18.119131 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78b780b8-88a8-40e8-a55e-cc60ed214515 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:18.119706 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78b780b8-88a8-40e8-a55e-cc60ed214515 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:18.120166 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78b780b8-88a8-40e8-a55e-cc60ed214515 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:18.120929 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78b780b8-88a8-40e8-a55e-cc60ed214515 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:18.121393 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78b780b8-88a8-40e8-a55e-cc60ed214515 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:18.121825 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78b780b8-88a8-40e8-a55e-cc60ed214515 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:18.199042 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-78b780b8-88a8-40e8-a55e-cc60ed214515 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:18.208788 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-78b780b8-88a8-40e8-a55e-cc60ed214515 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.102149 Aug 07 01:39:18.209452 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1204/2139] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:18 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:39:19.237581 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f34ebe40-a9b9-45cf-a621-96b6ffbd1054 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:19.238499 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f34ebe40-a9b9-45cf-a621-96b6ffbd1054 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:19.246310 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f34ebe40-a9b9-45cf-a621-96b6ffbd1054 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:19.246859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f34ebe40-a9b9-45cf-a621-96b6ffbd1054 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:19.247233 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f34ebe40-a9b9-45cf-a621-96b6ffbd1054 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:19.247952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f34ebe40-a9b9-45cf-a621-96b6ffbd1054 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:19.248435 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f34ebe40-a9b9-45cf-a621-96b6ffbd1054 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:19.248861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f34ebe40-a9b9-45cf-a621-96b6ffbd1054 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:19.331683 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f34ebe40-a9b9-45cf-a621-96b6ffbd1054 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:19.341178 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f34ebe40-a9b9-45cf-a621-96b6ffbd1054 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.107910 Aug 07 01:39:19.341772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 936/2140] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:19 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:39:20.362596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a8569f36-87df-4732-bdf7-39b30e300660 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:39:20.363243 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a8569f36-87df-4732-bdf7-39b30e300660 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] [instance: f481f4cc-2803-4f12-9cd2-db6055d1488c] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:39:20.370107 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8569f36-87df-4732-bdf7-39b30e300660 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:20.370597 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8569f36-87df-4732-bdf7-39b30e300660 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:20.371035 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8569f36-87df-4732-bdf7-39b30e300660 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:20.371994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8569f36-87df-4732-bdf7-39b30e300660 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:39:20.372504 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8569f36-87df-4732-bdf7-39b30e300660 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:39:20.372971 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a8569f36-87df-4732-bdf7-39b30e300660 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:39:20.441944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a8569f36-87df-4732-bdf7-39b30e300660 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6dd0ed2144164190b380ed58b3dd7cd4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f7b08cc6d4a34961bdf464b58946a02e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:39:20.451048 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a8569f36-87df-4732-bdf7-39b30e300660 tempest-VolumesSnapshotTestJSON-2086260047 tempest-VolumesSnapshotTestJSON-2086260047-project] 10.222.0.26 "GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c" status: 200 len: 1349 microversion: 2.1 time: 0.092430 Aug 07 01:39:20.451678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1205/2141] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:39:20 2022] GET /compute/v2.1/servers/f481f4cc-2803-4f12-9cd2-db6055d1488c => generated 1349 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:40:28.473222 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:40:28.474133 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:40:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:41:20.612480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR oslo.messaging._drivers.impl_rabbit [-] [503e4d92-a7e2-4c7e-a4e9-6cdd3fd4ae0d] AMQP server on 10.222.0.26:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Aug 07 01:41:20.613541 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=25 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:41:20.614061 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=77 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:41:20.620424 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-da5a514f-d73f-462c-9494-714ee2968f82 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:41:20.620974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-da5a514f-d73f-462c-9494-714ee2968f82 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:41:20.628609 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-da5a514f-d73f-462c-9494-714ee2968f82 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:41:20.628831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-da5a514f-d73f-462c-9494-714ee2968f82 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:41:20.629182 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-da5a514f-d73f-462c-9494-714ee2968f82 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:41:20.630303 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-da5a514f-d73f-462c-9494-714ee2968f82 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:41:20.631283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-da5a514f-d73f-462c-9494-714ee2968f82 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:41:20.631711 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-da5a514f-d73f-462c-9494-714ee2968f82 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:41:20.649290 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-da5a514f-d73f-462c-9494-714ee2968f82 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] HTTP exception thrown: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-da5a514f-d73f-462c-9494-714ee2968f82 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Request method failure captured: Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: DELETE /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be HTTP/1.1 Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xD021Vt2D0N359UOKX9xymFYpnZ79FQqLbXwvg5UbTRYuTTlR_HceEoHaOg5fvzy7nSB6FZNa7SVaU3q4lxIRVTXMupRK2mSDBaPiXGNUdpidSOmvxEoLqM4Zox-kqkoV9_K0_oxsudsRHb6jwecdh1rLe0262HFBUWOJmPZoVity8 Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:41:20.651183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:41:20.654116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:41:20.654116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:41:20.654116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:41:20.654116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:41:20.654116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:41:20.654116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:41:20.654116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:41:20.654116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: e65f0f1ba54f483abff788974b1256cb Aug 07 01:41:20.654116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:41:20.654116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:41:20.654116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:41:20.654116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '1143cfd8-c86b-4023-84cc-8518db39a1be'} Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:41:20.655480 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:41:20.659221 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:41:20.663000 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-da5a514f-d73f-462c-9494-714ee2968f82 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Returning 404 to user: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:41:20.663000 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-da5a514f-d73f-462c-9494-714ee2968f82 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "DELETE /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 404 len: 111 microversion: 2.1 time: 0.042614 Aug 07 01:41:20.663000 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 937/2142] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:41:20 2022] DELETE /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 07 01:41:20.663457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR oslo.messaging._drivers.impl_rabbit [-] [43d5fcde-35b3-4da6-9a89-8b1380fb4929] AMQP server on 10.222.0.26:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Aug 07 01:41:20.663457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=11 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:41:20.664883 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=47 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:41:20.670096 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e6e93d13-5c07-4094-843e-6ba3a2f5881c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:41:20.670858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e6e93d13-5c07-4094-843e-6ba3a2f5881c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 1143cfd8-c86b-4023-84cc-8518db39a1be] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:41:20.681744 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6e93d13-5c07-4094-843e-6ba3a2f5881c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:41:20.682400 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6e93d13-5c07-4094-843e-6ba3a2f5881c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:41:20.682966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6e93d13-5c07-4094-843e-6ba3a2f5881c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:41:20.683776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6e93d13-5c07-4094-843e-6ba3a2f5881c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:41:20.684385 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6e93d13-5c07-4094-843e-6ba3a2f5881c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:41:20.684941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e6e93d13-5c07-4094-843e-6ba3a2f5881c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:41:20.709325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-e6e93d13-5c07-4094-843e-6ba3a2f5881c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] HTTP exception thrown: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e6e93d13-5c07-4094-843e-6ba3a2f5881c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Request method failure captured: Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be HTTP/1.1 Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xD021Vt2D0N359UOKX9xymFYpnZ79FQqLbXwvg5UbTRYuTTlR_HceEoHaOg5fvzy7nSB6FZNa7SVaU3q4lxIRVTXMupRK2mSDBaPiXGNUdpidSOmvxEoLqM4Zox-kqkoV9_K0_oxsudsRHb6jwecdh1rLe0262HFBUWOJmPZoVity8 Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:41:20.711512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:41:20.714275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:41:20.714275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:41:20.714275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:41:20.714275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:41:20.714275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:41:20.714275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:41:20.714275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:41:20.714275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: e65f0f1ba54f483abff788974b1256cb Aug 07 01:41:20.714275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:41:20.714275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:41:20.714275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:41:20.714275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '1143cfd8-c86b-4023-84cc-8518db39a1be'} Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:41:20.716632 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. Aug 07 01:41:20.719972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:41:20.722315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e6e93d13-5c07-4094-843e-6ba3a2f5881c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Returning 404 to user: Instance 1143cfd8-c86b-4023-84cc-8518db39a1be could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:41:20.722315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e6e93d13-5c07-4094-843e-6ba3a2f5881c tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be" status: 404 len: 111 microversion: 2.1 time: 0.054415 Aug 07 01:41:20.722315 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1206/2143] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:41:20 2022] GET /compute/v2.1/servers/1143cfd8-c86b-4023-84cc-8518db39a1be => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:41:48.046717 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:41:48.047380 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:41:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:41:54.249968 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:41:54.250653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:41:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:42:04.558795 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:42:04.559891 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:42:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:42:26.986586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:42:26.987240 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:42:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:42:28.051497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:42:28.052154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:42:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:43:48.484823 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:43:48.485806 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:43:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:45:08.055889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:45:08.056828 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:45:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:45:14.260514 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:45:14.261683 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:45:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:45:24.568702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:45:24.569604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:45:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:45:46.996628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:45:46.997289 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:45:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:45:48.061279 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:45:48.061941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:45:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:46:21.676590 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:46:21.677928 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:46:21.687861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-752eab13-dafd-4fc6-af4f-2373bbac5b6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:46:21.687861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-752eab13-dafd-4fc6-af4f-2373bbac5b6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:46:21.701967 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] [503e4d92-a7e2-4c7e-a4e9-6cdd3fd4ae0d] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 41086. Aug 07 01:46:21.707835 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-752eab13-dafd-4fc6-af4f-2373bbac5b6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:21.708601 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-752eab13-dafd-4fc6-af4f-2373bbac5b6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:21.709065 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-752eab13-dafd-4fc6-af4f-2373bbac5b6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:21.709826 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-752eab13-dafd-4fc6-af4f-2373bbac5b6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:21.710297 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-752eab13-dafd-4fc6-af4f-2373bbac5b6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:21.711392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-752eab13-dafd-4fc6-af4f-2373bbac5b6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:21.742158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-752eab13-dafd-4fc6-af4f-2373bbac5b6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Going to try to terminate instance {{(pid=117984) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:46:21.874767 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-752eab13-dafd-4fc6-af4f-2373bbac5b6d tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 204 len: 0 microversion: 2.1 time: 0.201540 Aug 07 01:46:21.875240 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 938/2144] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:46:21 2022] DELETE /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 07 01:46:21.886950 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:46:21.890492 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=2 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:46:21.899833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8c8257d1-c199-4732-adeb-de1c38b06728 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:46:21.900999 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8c8257d1-c199-4732-adeb-de1c38b06728 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:46:21.913063 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] [43d5fcde-35b3-4da6-9a89-8b1380fb4929] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 41094. Aug 07 01:46:21.921463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c8257d1-c199-4732-adeb-de1c38b06728 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:21.922258 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c8257d1-c199-4732-adeb-de1c38b06728 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:21.923721 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c8257d1-c199-4732-adeb-de1c38b06728 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:21.924961 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c8257d1-c199-4732-adeb-de1c38b06728 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:21.925648 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c8257d1-c199-4732-adeb-de1c38b06728 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:21.926196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8c8257d1-c199-4732-adeb-de1c38b06728 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:22.470612 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8c8257d1-c199-4732-adeb-de1c38b06728 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:46:22.480368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8c8257d1-c199-4732-adeb-de1c38b06728 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1508 microversion: 2.1 time: 0.598450 Aug 07 01:46:22.488060 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1207/2145] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:46:21 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1508 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:46:23.504564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-955ff7b1-3b21-4c2f-96e5-51d1745d5289 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:46:23.505771 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-955ff7b1-3b21-4c2f-96e5-51d1745d5289 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:46:23.513868 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-955ff7b1-3b21-4c2f-96e5-51d1745d5289 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:23.514745 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-955ff7b1-3b21-4c2f-96e5-51d1745d5289 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:23.515539 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-955ff7b1-3b21-4c2f-96e5-51d1745d5289 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:23.517386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-955ff7b1-3b21-4c2f-96e5-51d1745d5289 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:23.518259 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-955ff7b1-3b21-4c2f-96e5-51d1745d5289 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:23.519934 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-955ff7b1-3b21-4c2f-96e5-51d1745d5289 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:24.035889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-955ff7b1-3b21-4c2f-96e5-51d1745d5289 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:46:24.050919 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-955ff7b1-3b21-4c2f-96e5-51d1745d5289 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1508 microversion: 2.1 time: 0.551565 Aug 07 01:46:24.052151 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 939/2146] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:46:23 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1508 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:46:25.081148 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-3078c253-4455-4874-aa6f-b0c2cceea673 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:46:25.081148 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-3078c253-4455-4874-aa6f-b0c2cceea673 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:46:25.089436 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3078c253-4455-4874-aa6f-b0c2cceea673 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:25.090515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3078c253-4455-4874-aa6f-b0c2cceea673 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:25.091328 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3078c253-4455-4874-aa6f-b0c2cceea673 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:25.092754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3078c253-4455-4874-aa6f-b0c2cceea673 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:25.093674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3078c253-4455-4874-aa6f-b0c2cceea673 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:25.095047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-3078c253-4455-4874-aa6f-b0c2cceea673 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:25.355836 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-3078c253-4455-4874-aa6f-b0c2cceea673 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:46:25.369696 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3078c253-4455-4874-aa6f-b0c2cceea673 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1508 microversion: 2.1 time: 0.295698 Aug 07 01:46:25.371142 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1208/2147] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:46:25 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1508 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:46:26.414314 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7ce8dd36-d573-4361-aaf7-1131593ad4d0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:46:26.414984 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7ce8dd36-d573-4361-aaf7-1131593ad4d0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:46:26.421991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7ce8dd36-d573-4361-aaf7-1131593ad4d0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:26.422989 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7ce8dd36-d573-4361-aaf7-1131593ad4d0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:26.423580 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7ce8dd36-d573-4361-aaf7-1131593ad4d0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:26.424533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7ce8dd36-d573-4361-aaf7-1131593ad4d0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:26.425031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7ce8dd36-d573-4361-aaf7-1131593ad4d0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:26.425432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7ce8dd36-d573-4361-aaf7-1131593ad4d0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:26.610608 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7ce8dd36-d573-4361-aaf7-1131593ad4d0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:46:26.619338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7ce8dd36-d573-4361-aaf7-1131593ad4d0 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1508 microversion: 2.1 time: 0.209229 Aug 07 01:46:26.619945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 940/2148] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:46:26 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1508 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:46:27.657930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-73a722c9-042f-4058-9fc7-c2a876595661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:46:27.659163 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-73a722c9-042f-4058-9fc7-c2a876595661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:46:27.673404 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73a722c9-042f-4058-9fc7-c2a876595661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:27.674353 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73a722c9-042f-4058-9fc7-c2a876595661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:27.675144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73a722c9-042f-4058-9fc7-c2a876595661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:27.676758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73a722c9-042f-4058-9fc7-c2a876595661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:27.677572 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73a722c9-042f-4058-9fc7-c2a876595661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:27.678450 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-73a722c9-042f-4058-9fc7-c2a876595661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:27.957865 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-73a722c9-042f-4058-9fc7-c2a876595661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:46:27.974199 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-73a722c9-042f-4058-9fc7-c2a876595661 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1508 microversion: 2.1 time: 0.326678 Aug 07 01:46:27.989213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1209/2149] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:46:27 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1508 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:46:28.877261 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-c0aae125-3c0b-4189-bf84-58405faf4247 req-5eff5f7a-be69-4e17-a548-8a7859f52370 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3b71d249-75ce-4352-bb93-34e88cc0df93", "name": "network-vif-deleted", "tag": "9ea29730-37e9-45db-98d4-1ef7d39b89d9"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:46:28.889319 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c0aae125-3c0b-4189-bf84-58405faf4247 req-5eff5f7a-be69-4e17-a548-8a7859f52370 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:28.890154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c0aae125-3c0b-4189-bf84-58405faf4247 req-5eff5f7a-be69-4e17-a548-8a7859f52370 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:28.891001 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c0aae125-3c0b-4189-bf84-58405faf4247 req-5eff5f7a-be69-4e17-a548-8a7859f52370 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:28.904369 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-c0aae125-3c0b-4189-bf84-58405faf4247 req-5eff5f7a-be69-4e17-a548-8a7859f52370 service nova] Creating event network-vif-deleted:9ea29730-37e9-45db-98d4-1ef7d39b89d9 for instance 3b71d249-75ce-4352-bb93-34e88cc0df93 on hv-cinder-maste Aug 07 01:46:28.911386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-c0aae125-3c0b-4189-bf84-58405faf4247 req-5eff5f7a-be69-4e17-a548-8a7859f52370 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.495070 Aug 07 01:46:28.912773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 941/2150] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:46:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:46:29.018017 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b277af88-f9e9-4694-bd25-f1d8f3bab1de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:46:29.018953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b277af88-f9e9-4694-bd25-f1d8f3bab1de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:46:29.030989 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b277af88-f9e9-4694-bd25-f1d8f3bab1de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:29.033251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b277af88-f9e9-4694-bd25-f1d8f3bab1de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:29.042226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b277af88-f9e9-4694-bd25-f1d8f3bab1de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:29.043308 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b277af88-f9e9-4694-bd25-f1d8f3bab1de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:29.043781 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b277af88-f9e9-4694-bd25-f1d8f3bab1de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:29.045520 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b277af88-f9e9-4694-bd25-f1d8f3bab1de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:29.095251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b277af88-f9e9-4694-bd25-f1d8f3bab1de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:29.095857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b277af88-f9e9-4694-bd25-f1d8f3bab1de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:29.096917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b277af88-f9e9-4694-bd25-f1d8f3bab1de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:29.097585 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.objects.instance [None req-b277af88-f9e9-4694-bd25-f1d8f3bab1de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lazy-loading 'fault' on Instance uuid 3b71d249-75ce-4352-bb93-34e88cc0df93 {{(pid=117983) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:46:29.168050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-b277af88-f9e9-4694-bd25-f1d8f3bab1de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:46:29.178024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b277af88-f9e9-4694-bd25-f1d8f3bab1de tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 200 len: 1312 microversion: 2.1 time: 0.174289 Aug 07 01:46:29.192103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1210/2151] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:46:29 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 1312 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:46:30.200266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cf3453a5-f4b6-4fb4-b44e-c15351a04b47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:46:30.200994 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-cf3453a5-f4b6-4fb4-b44e-c15351a04b47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 3b71d249-75ce-4352-bb93-34e88cc0df93] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:46:30.208158 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf3453a5-f4b6-4fb4-b44e-c15351a04b47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:30.208680 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf3453a5-f4b6-4fb4-b44e-c15351a04b47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:30.209144 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf3453a5-f4b6-4fb4-b44e-c15351a04b47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:30.209894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf3453a5-f4b6-4fb4-b44e-c15351a04b47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:30.210355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf3453a5-f4b6-4fb4-b44e-c15351a04b47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:30.210821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cf3453a5-f4b6-4fb4-b44e-c15351a04b47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:30.227907 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-cf3453a5-f4b6-4fb4-b44e-c15351a04b47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] HTTP exception thrown: Instance 3b71d249-75ce-4352-bb93-34e88cc0df93 could not be found. Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cf3453a5-f4b6-4fb4-b44e-c15351a04b47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Request method failure captured: Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 HTTP/1.1 Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xFan_-oNUM4AqSt7Wni9p606OoWcdJLz4SIr5VHLwgODJGL0BBSKB64xI7IBMYJhEzGuAdjPCyw6B3DmJ9opPnSTNGpjpiLZ8MHcy7T0m2nWHFImY2c23r73oTOGBUNFDYYRljAnrO4awdzCByqv2_7MGWKYhYTsQ-EtnOqUmEQZ_Q Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 9badaa18dcf64810b10dd7bf03d8e40d Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1786214574 Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:46:30.229675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:46:30.231634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:46:30.231634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1786214574 Aug 07 01:46:30.231634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 9badaa18dcf64810b10dd7bf03d8e40d Aug 07 01:46:30.231634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1786214574 Aug 07 01:46:30.231634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-ServerRescueNegativeTestJSON-1786214574-project Aug 07 01:46:30.231634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:46:30.231634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:46:30.231634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: ec931c9cc56a462991413926d34dfa46 Aug 07 01:46:30.231634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1786214574-project Aug 07 01:46:30.231634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:46:30.231634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 3b71d249-75ce-4352-bb93-34e88cc0df93 could not be found. Aug 07 01:46:30.231634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '3b71d249-75ce-4352-bb93-34e88cc0df93'} Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3b71d249-75ce-4352-bb93-34e88cc0df93 could not be found. Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:46:30.232796 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3b71d249-75ce-4352-bb93-34e88cc0df93 could not be found. Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3b71d249-75ce-4352-bb93-34e88cc0df93 could not be found. Aug 07 01:46:30.234321 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:46:30.236785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cf3453a5-f4b6-4fb4-b44e-c15351a04b47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Returning 404 to user: Instance 3b71d249-75ce-4352-bb93-34e88cc0df93 could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:46:30.236785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-cf3453a5-f4b6-4fb4-b44e-c15351a04b47 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93" status: 404 len: 111 microversion: 2.1 time: 0.035789 Aug 07 01:46:30.236785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 942/2152] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:46:30 2022] GET /compute/v2.1/servers/3b71d249-75ce-4352-bb93-34e88cc0df93 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 07 01:46:30.240596 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e36ac41b-7d60-4364-959f-104bf0bfbafb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:46:30.241122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e36ac41b-7d60-4364-959f-104bf0bfbafb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:46:30.247684 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e36ac41b-7d60-4364-959f-104bf0bfbafb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:30.248168 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e36ac41b-7d60-4364-959f-104bf0bfbafb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:30.248656 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e36ac41b-7d60-4364-959f-104bf0bfbafb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:30.249374 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e36ac41b-7d60-4364-959f-104bf0bfbafb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:30.249833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e36ac41b-7d60-4364-959f-104bf0bfbafb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:30.250249 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e36ac41b-7d60-4364-959f-104bf0bfbafb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:30.289696 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e36ac41b-7d60-4364-959f-104bf0bfbafb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Going to try to terminate instance {{(pid=117983) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:46:30.392698 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e36ac41b-7d60-4364-959f-104bf0bfbafb tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "DELETE /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 204 len: 0 microversion: 2.1 time: 0.155508 Aug 07 01:46:30.393290 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1211/2153] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:46:30 2022] DELETE /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 07 01:46:30.402346 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e51502bc-26cd-46b2-a931-becdf4c103c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:46:30.402985 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e51502bc-26cd-46b2-a931-becdf4c103c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:46:30.409831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e51502bc-26cd-46b2-a931-becdf4c103c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:30.410239 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e51502bc-26cd-46b2-a931-becdf4c103c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:30.410675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e51502bc-26cd-46b2-a931-becdf4c103c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:30.411551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e51502bc-26cd-46b2-a931-becdf4c103c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:30.411832 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e51502bc-26cd-46b2-a931-becdf4c103c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:30.412286 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e51502bc-26cd-46b2-a931-becdf4c103c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:30.615599 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e51502bc-26cd-46b2-a931-becdf4c103c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:46:30.624376 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e51502bc-26cd-46b2-a931-becdf4c103c1 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1523 microversion: 2.1 time: 0.226220 Aug 07 01:46:30.625302 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 943/2154] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:46:30 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1523 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:46:31.642919 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-fd68ac52-41d8-49a9-994c-a3959b97b758 req-6da57634-05a1-4e05-a195-2bab7ce5b616 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "99e925dd-a3a5-44ab-bf20-9c80807d5530", "name": "network-vif-deleted", "tag": "c91dc146-cf64-40b0-a3e9-644405de258c"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:46:31.655494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-fd68ac52-41d8-49a9-994c-a3959b97b758 req-6da57634-05a1-4e05-a195-2bab7ce5b616 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:31.656036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-fd68ac52-41d8-49a9-994c-a3959b97b758 req-6da57634-05a1-4e05-a195-2bab7ce5b616 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:31.656527 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-fd68ac52-41d8-49a9-994c-a3959b97b758 req-6da57634-05a1-4e05-a195-2bab7ce5b616 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:31.670388 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-fd68ac52-41d8-49a9-994c-a3959b97b758 req-6da57634-05a1-4e05-a195-2bab7ce5b616 service nova] Creating event network-vif-deleted:c91dc146-cf64-40b0-a3e9-644405de258c for instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 on hv-cinder-maste Aug 07 01:46:31.673495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-edec18c0-4ba2-471c-a831-eaca5f4acc05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:46:31.676333 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-fd68ac52-41d8-49a9-994c-a3959b97b758 req-6da57634-05a1-4e05-a195-2bab7ce5b616 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038675 Aug 07 01:46:31.677245 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1212/2155] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:46:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:46:31.678958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-edec18c0-4ba2-471c-a831-eaca5f4acc05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:46:31.702239 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-edec18c0-4ba2-471c-a831-eaca5f4acc05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:31.703104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-edec18c0-4ba2-471c-a831-eaca5f4acc05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:31.704400 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-edec18c0-4ba2-471c-a831-eaca5f4acc05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:31.705696 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-edec18c0-4ba2-471c-a831-eaca5f4acc05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:31.706523 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-edec18c0-4ba2-471c-a831-eaca5f4acc05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:31.707289 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-edec18c0-4ba2-471c-a831-eaca5f4acc05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:31.831507 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-edec18c0-4ba2-471c-a831-eaca5f4acc05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ec931c9cc56a462991413926d34dfa46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9badaa18dcf64810b10dd7bf03d8e40d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:46:31.839803 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-edec18c0-4ba2-471c-a831-eaca5f4acc05 tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 200 len: 1307 microversion: 2.1 time: 0.193307 Aug 07 01:46:31.841055 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 944/2156] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:46:31 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 1307 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:46:32.895663 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-51e52655-9713-4611-9f7d-27ac66e2e54b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:46:32.900864 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-51e52655-9713-4611-9f7d-27ac66e2e54b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] [instance: 99e925dd-a3a5-44ab-bf20-9c80807d5530] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:46:32.931566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51e52655-9713-4611-9f7d-27ac66e2e54b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:32.933451 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51e52655-9713-4611-9f7d-27ac66e2e54b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:32.934076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51e52655-9713-4611-9f7d-27ac66e2e54b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:32.935047 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51e52655-9713-4611-9f7d-27ac66e2e54b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:46:32.936189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51e52655-9713-4611-9f7d-27ac66e2e54b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:46:32.939300 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51e52655-9713-4611-9f7d-27ac66e2e54b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:46:32.970155 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-51e52655-9713-4611-9f7d-27ac66e2e54b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] HTTP exception thrown: Instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 could not be found. Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-51e52655-9713-4611-9f7d-27ac66e2e54b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Request method failure captured: Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 HTTP/1.1 Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xFan_-oNUM4AqSt7Wni9p606OoWcdJLz4SIr5VHLwgODJGL0BBSKB64xI7IBMYJhEzGuAdjPCyw6B3DmJ9opPnSTNGpjpiLZ8MHcy7T0m2nWHFImY2c23r73oTOGBUNFDYYRljAnrO4awdzCByqv2_7MGWKYhYTsQ-EtnOqUmEQZ_Q Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 9badaa18dcf64810b10dd7bf03d8e40d Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1786214574 Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:46:32.974666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:46:32.976497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9badaa18dcf64810b10dd7bf03d8e40d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:46:32.976497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1786214574 Aug 07 01:46:32.976497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 9badaa18dcf64810b10dd7bf03d8e40d Aug 07 01:46:32.976497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1786214574 Aug 07 01:46:32.976497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-ServerRescueNegativeTestJSON-1786214574-project Aug 07 01:46:32.976497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:46:32.976497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:46:32.976497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: ec931c9cc56a462991413926d34dfa46 Aug 07 01:46:32.976497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1786214574-project Aug 07 01:46:32.976497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:46:32.976497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 could not be found. Aug 07 01:46:32.976497 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '99e925dd-a3a5-44ab-bf20-9c80807d5530'} Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 could not be found. Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:46:32.977532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 could not be found. Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 could not be found. Aug 07 01:46:32.979679 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:46:32.981464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-51e52655-9713-4611-9f7d-27ac66e2e54b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] Returning 404 to user: Instance 99e925dd-a3a5-44ab-bf20-9c80807d5530 could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:46:32.981464 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-51e52655-9713-4611-9f7d-27ac66e2e54b tempest-ServerRescueNegativeTestJSON-1786214574 tempest-ServerRescueNegativeTestJSON-1786214574-project] 10.222.0.26 "GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530" status: 404 len: 111 microversion: 2.1 time: 0.099766 Aug 07 01:46:32.988539 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1213/2157] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:46:32 2022] GET /compute/v2.1/servers/99e925dd-a3a5-44ab-bf20-9c80807d5530 => generated 111 bytes in 112 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 07 01:47:08.496186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:47:08.496899 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:47:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:47:20.714429 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9ee856dc-7dc0-447c-ad78-21de961543b5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:47:20.715103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9ee856dc-7dc0-447c-ad78-21de961543b5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:47:20.717670 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=19,nova_api:UPDATE=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:47:20.718567 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=42,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:47:20.731240 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ee856dc-7dc0-447c-ad78-21de961543b5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:47:20.731974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ee856dc-7dc0-447c-ad78-21de961543b5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:47:20.732628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ee856dc-7dc0-447c-ad78-21de961543b5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:47:20.733653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ee856dc-7dc0-447c-ad78-21de961543b5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:47:20.734454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ee856dc-7dc0-447c-ad78-21de961543b5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:47:20.735434 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ee856dc-7dc0-447c-ad78-21de961543b5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:47:20.754419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-9ee856dc-7dc0-447c-ad78-21de961543b5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] HTTP exception thrown: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9ee856dc-7dc0-447c-ad78-21de961543b5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Request method failure captured: Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: DELETE /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 HTTP/1.1 Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xD021Vt2D0N359UOKX9xymFYpnZ79FQqLbXwvg5UbTRYuTTlR_HceEoHaOg5fvzy7nSB6FZNa7SVaU3q4lxIRVTXMupRK2mSDBaPiXGNUdpidSOmvxEoLqM4Zox-kqkoV9_K0_oxsudsRHb6jwecdh1rLe0262HFBUWOJmPZoVity8 Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:47:20.756018 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:47:20.757270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:47:20.757270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:20.757270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:47:20.757270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:20.757270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:47:20.757270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:47:20.757270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:47:20.757270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: e65f0f1ba54f483abff788974b1256cb Aug 07 01:47:20.757270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:47:20.757270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:47:20.757270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:47:20.757270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': 'e86efc1a-b689-454a-8683-526a9c18e222'} Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:47:20.757853 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:47:20.762383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.763280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9ee856dc-7dc0-447c-ad78-21de961543b5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Returning 404 to user: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:47:20.763280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9ee856dc-7dc0-447c-ad78-21de961543b5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 404 len: 111 microversion: 2.1 time: 0.048086 Aug 07 01:47:20.763280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 945/2158] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:47:20 2022] DELETE /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 07 01:47:20.768554 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f2495f36-2a3b-4350-aa4f-6d66696d3cb5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:47:20.769177 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f2495f36-2a3b-4350-aa4f-6d66696d3cb5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: e86efc1a-b689-454a-8683-526a9c18e222] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:47:20.771444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=21,nova_api:UPDATE=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:47:20.772113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=44,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:47:20.779245 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2495f36-2a3b-4350-aa4f-6d66696d3cb5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:47:20.779740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2495f36-2a3b-4350-aa4f-6d66696d3cb5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:47:20.780182 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2495f36-2a3b-4350-aa4f-6d66696d3cb5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:47:20.781029 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2495f36-2a3b-4350-aa4f-6d66696d3cb5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:47:20.781488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2495f36-2a3b-4350-aa4f-6d66696d3cb5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:47:20.781973 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f2495f36-2a3b-4350-aa4f-6d66696d3cb5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:47:20.801041 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-f2495f36-2a3b-4350-aa4f-6d66696d3cb5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] HTTP exception thrown: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f2495f36-2a3b-4350-aa4f-6d66696d3cb5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Request method failure captured: Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 HTTP/1.1 Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xD021Vt2D0N359UOKX9xymFYpnZ79FQqLbXwvg5UbTRYuTTlR_HceEoHaOg5fvzy7nSB6FZNa7SVaU3q4lxIRVTXMupRK2mSDBaPiXGNUdpidSOmvxEoLqM4Zox-kqkoV9_K0_oxsudsRHb6jwecdh1rLe0262HFBUWOJmPZoVity8 Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:47:20.802720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:47:20.804669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:47:20.804669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:20.804669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:47:20.804669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:20.804669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:47:20.804669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:47:20.804669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:47:20.804669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: e65f0f1ba54f483abff788974b1256cb Aug 07 01:47:20.804669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:47:20.804669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:47:20.804669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:47:20.804669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': 'e86efc1a-b689-454a-8683-526a9c18e222'} Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:47:20.806021 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. Aug 07 01:47:20.808138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.810273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f2495f36-2a3b-4350-aa4f-6d66696d3cb5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Returning 404 to user: Instance e86efc1a-b689-454a-8683-526a9c18e222 could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:47:20.810273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f2495f36-2a3b-4350-aa4f-6d66696d3cb5 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222" status: 404 len: 111 microversion: 2.1 time: 0.041214 Aug 07 01:47:20.810273 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1214/2159] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:47:20 2022] GET /compute/v2.1/servers/e86efc1a-b689-454a-8683-526a9c18e222 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:47:20.815384 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-547f44e7-17c2-422a-8098-64a0ddbed6db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:47:20.815986 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-547f44e7-17c2-422a-8098-64a0ddbed6db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:47:20.822865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-547f44e7-17c2-422a-8098-64a0ddbed6db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:47:20.823373 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-547f44e7-17c2-422a-8098-64a0ddbed6db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:47:20.823935 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-547f44e7-17c2-422a-8098-64a0ddbed6db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:47:20.824573 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-547f44e7-17c2-422a-8098-64a0ddbed6db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:47:20.825035 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-547f44e7-17c2-422a-8098-64a0ddbed6db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:47:20.825464 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-547f44e7-17c2-422a-8098-64a0ddbed6db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:47:20.842692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-547f44e7-17c2-422a-8098-64a0ddbed6db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] HTTP exception thrown: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-547f44e7-17c2-422a-8098-64a0ddbed6db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Request method failure captured: Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: DELETE /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc HTTP/1.1 Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xD021Vt2D0N359UOKX9xymFYpnZ79FQqLbXwvg5UbTRYuTTlR_HceEoHaOg5fvzy7nSB6FZNa7SVaU3q4lxIRVTXMupRK2mSDBaPiXGNUdpidSOmvxEoLqM4Zox-kqkoV9_K0_oxsudsRHb6jwecdh1rLe0262HFBUWOJmPZoVity8 Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:47:20.844218 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:47:20.845447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:47:20.845447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:20.845447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:47:20.845447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:20.845447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:47:20.845447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:47:20.845447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:47:20.845447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: e65f0f1ba54f483abff788974b1256cb Aug 07 01:47:20.845447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:47:20.845447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:47:20.845447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:47:20.845447 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '54a16243-ecd7-4152-a1d3-bf92cc3709cc'} Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:47:20.846121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:47:20.847212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.848662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-547f44e7-17c2-422a-8098-64a0ddbed6db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Returning 404 to user: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:47:20.848662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-547f44e7-17c2-422a-8098-64a0ddbed6db tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "DELETE /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 404 len: 111 microversion: 2.1 time: 0.035086 Aug 07 01:47:20.848662 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 946/2160] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:47:20 2022] DELETE /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:47:20.856655 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-70c44692-fa11-4d0d-b98c-ea84976f9c2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:47:20.857321 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-70c44692-fa11-4d0d-b98c-ea84976f9c2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 54a16243-ecd7-4152-a1d3-bf92cc3709cc] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:47:20.865271 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-70c44692-fa11-4d0d-b98c-ea84976f9c2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:47:20.865771 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-70c44692-fa11-4d0d-b98c-ea84976f9c2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:47:20.866220 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-70c44692-fa11-4d0d-b98c-ea84976f9c2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:47:20.866945 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-70c44692-fa11-4d0d-b98c-ea84976f9c2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:47:20.867401 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-70c44692-fa11-4d0d-b98c-ea84976f9c2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:47:20.867832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-70c44692-fa11-4d0d-b98c-ea84976f9c2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:47:20.882984 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-70c44692-fa11-4d0d-b98c-ea84976f9c2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] HTTP exception thrown: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-70c44692-fa11-4d0d-b98c-ea84976f9c2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Request method failure captured: Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc HTTP/1.1 Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xD021Vt2D0N359UOKX9xymFYpnZ79FQqLbXwvg5UbTRYuTTlR_HceEoHaOg5fvzy7nSB6FZNa7SVaU3q4lxIRVTXMupRK2mSDBaPiXGNUdpidSOmvxEoLqM4Zox-kqkoV9_K0_oxsudsRHb6jwecdh1rLe0262HFBUWOJmPZoVity8 Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:47:20.884590 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:47:20.885734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:47:20.885734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:20.885734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:47:20.885734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:20.885734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:47:20.885734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:47:20.885734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:47:20.885734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: e65f0f1ba54f483abff788974b1256cb Aug 07 01:47:20.885734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:47:20.885734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:47:20.885734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:47:20.885734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '54a16243-ecd7-4152-a1d3-bf92cc3709cc'} Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:47:20.886422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. Aug 07 01:47:20.887354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:20.888145 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-70c44692-fa11-4d0d-b98c-ea84976f9c2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Returning 404 to user: Instance 54a16243-ecd7-4152-a1d3-bf92cc3709cc could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:47:20.888145 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-70c44692-fa11-4d0d-b98c-ea84976f9c2d tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc" status: 404 len: 111 microversion: 2.1 time: 0.034565 Aug 07 01:47:20.888145 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1215/2161] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:47:20 2022] GET /compute/v2.1/servers/54a16243-ecd7-4152-a1d3-bf92cc3709cc => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 07 01:47:25.227882 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-105002df-e590-45b5-9d0f-2be19b1dea94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:47:25.228818 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-105002df-e590-45b5-9d0f-2be19b1dea94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:47:25.240263 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-105002df-e590-45b5-9d0f-2be19b1dea94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:47:25.241103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-105002df-e590-45b5-9d0f-2be19b1dea94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:47:25.241801 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-105002df-e590-45b5-9d0f-2be19b1dea94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:47:25.243015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-105002df-e590-45b5-9d0f-2be19b1dea94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:47:25.243813 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-105002df-e590-45b5-9d0f-2be19b1dea94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:47:25.244532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-105002df-e590-45b5-9d0f-2be19b1dea94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:47:25.265167 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-105002df-e590-45b5-9d0f-2be19b1dea94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] HTTP exception thrown: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-105002df-e590-45b5-9d0f-2be19b1dea94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Request method failure captured: Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: DELETE /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee HTTP/1.1 Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xD021Vt2D0N359UOKX9xymFYpnZ79FQqLbXwvg5UbTRYuTTlR_HceEoHaOg5fvzy7nSB6FZNa7SVaU3q4lxIRVTXMupRK2mSDBaPiXGNUdpidSOmvxEoLqM4Zox-kqkoV9_K0_oxsudsRHb6jwecdh1rLe0262HFBUWOJmPZoVity8 Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:47:25.267280 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:47:25.268959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:47:25.268959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:25.268959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:47:25.268959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:25.268959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:47:25.268959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:47:25.268959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:47:25.268959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: e65f0f1ba54f483abff788974b1256cb Aug 07 01:47:25.268959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:47:25.268959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:47:25.268959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:47:25.268959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '96f9a9ce-f29e-48be-a36b-8518126a09ee'} Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:47:25.269533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:47:25.271443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:25.273594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-105002df-e590-45b5-9d0f-2be19b1dea94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Returning 404 to user: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:47:25.273594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-105002df-e590-45b5-9d0f-2be19b1dea94 tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "DELETE /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 404 len: 111 microversion: 2.1 time: 0.049003 Aug 07 01:47:25.273594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 947/2162] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:47:25 2022] DELETE /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:47:25.283480 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b94b61d8-78e5-44ea-9a20-feb994b0558e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:47:25.284699 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-b94b61d8-78e5-44ea-9a20-feb994b0558e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] [instance: 96f9a9ce-f29e-48be-a36b-8518126a09ee] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:47:25.292938 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b94b61d8-78e5-44ea-9a20-feb994b0558e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:47:25.293305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b94b61d8-78e5-44ea-9a20-feb994b0558e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:47:25.293759 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b94b61d8-78e5-44ea-9a20-feb994b0558e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:47:25.294502 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b94b61d8-78e5-44ea-9a20-feb994b0558e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:47:25.294978 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b94b61d8-78e5-44ea-9a20-feb994b0558e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:47:25.295407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-b94b61d8-78e5-44ea-9a20-feb994b0558e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:47:25.312313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-b94b61d8-78e5-44ea-9a20-feb994b0558e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] HTTP exception thrown: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b94b61d8-78e5-44ea-9a20-feb994b0558e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Request method failure captured: Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee HTTP/1.1 Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xD021Vt2D0N359UOKX9xymFYpnZ79FQqLbXwvg5UbTRYuTTlR_HceEoHaOg5fvzy7nSB6FZNa7SVaU3q4lxIRVTXMupRK2mSDBaPiXGNUdpidSOmvxEoLqM4Zox-kqkoV9_K0_oxsudsRHb6jwecdh1rLe0262HFBUWOJmPZoVity8 Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:47:25.313969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:47:25.315800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/35820b4f79774ac49f43fce915f56d6f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:47:25.315800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:25.315800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 35820b4f79774ac49f43fce915f56d6f Aug 07 01:47:25.315800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-383374120 Aug 07 01:47:25.315800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:47:25.315800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:47:25.315800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:47:25.315800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: e65f0f1ba54f483abff788974b1256cb Aug 07 01:47:25.315800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-AttachVolumeNegativeTest-383374120-project Aug 07 01:47:25.315800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:47:25.315800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:47:25.315800 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '96f9a9ce-f29e-48be-a36b-8518126a09ee'} Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:47:25.316694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. Aug 07 01:47:25.317650 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:25.318542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b94b61d8-78e5-44ea-9a20-feb994b0558e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] Returning 404 to user: Instance 96f9a9ce-f29e-48be-a36b-8518126a09ee could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:47:25.318542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b94b61d8-78e5-44ea-9a20-feb994b0558e tempest-AttachVolumeNegativeTest-383374120 tempest-AttachVolumeNegativeTest-383374120-project] 10.222.0.26 "GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee" status: 404 len: 111 microversion: 2.1 time: 0.039653 Aug 07 01:47:25.318542 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1216/2163] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:47:25 2022] GET /compute/v2.1/servers/96f9a9ce-f29e-48be-a36b-8518126a09ee => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:47:31.710611 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-432ffe4f-382c-4785-9817-c48927fc986d tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Action: 'create', calling method: .version_select at 0x7f1fad1a8c10>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1945912385", "metadata": {"Type": "work"}}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-432ffe4f-382c-4785-9817-c48927fc986d tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Request method failure captured: Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 122 Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xmzGzN3tL5wLhfdjWHcU9TVXMQG8bLLObeSkPyWZKGoyxJrWuiqBPErSYAes88wSM_NEi6z-7-p-Q9magtURjPNOt4NZWLpO3o7TcmfniIY2cVyZr27F2qKCVwXZdiSk0E2ac4d8eLDxhUZU5vceTQYnuhBNmm06AewOfFuCmOlrjk Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: c8597a47523048e78c0791720c6e677a Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:47:31.726250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:47:31.728504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c8597a47523048e78c0791720c6e677a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c8597a47523048e78c0791720c6e677a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c8597a47523048e78c0791720c6e677a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c8597a47523048e78c0791720c6e677a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:47:31.728504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.728504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: c8597a47523048e78c0791720c6e677a Aug 07 01:47:31.728504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.728504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-VolumesNegativeTest-1563497495-project Aug 07 01:47:31.728504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:47:31.728504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:47:31.728504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: 3466f9e75a644b04869bf7f460b8642e Aug 07 01:47:31.728504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-VolumesNegativeTest-1563497495-project Aug 07 01:47:31.728504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Aug 07 01:47:31.728504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1945912385", "metadata": {"Type": "work"}}} Aug 07 01:47:31.728504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: Aug 07 01:47:31.728504 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1945912385', 'metadata': {'Type': 'work'}}}} Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise error Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi '#$%' Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:47:31.729693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 07 01:47:31.731092 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 07 01:47:31.732991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 07 01:47:31.732991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.732991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-432ffe4f-382c-4785-9817-c48927fc986d tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:47:31.732991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-432ffe4f-382c-4785-9817-c48927fc986d tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.202324 Aug 07 01:47:31.732991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 948/2164] 10.222.0.26 () {60 vars in 1156 bytes} [Sun Aug 7 01:47:31 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 204 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Aug 07 01:47:31.749295 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c6c823c1-7dc5-4c34-890f-ccda1a376b63 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Action: 'create', calling method: .version_select at 0x7f1fad116f70>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-65803002", "metadata": {"Type": "work"}}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c6c823c1-7dc5-4c34-890f-ccda1a376b63 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Request method failure captured: Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 118 Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xmzGzN3tL5wLhfdjWHcU9TVXMQG8bLLObeSkPyWZKGoyxJrWuiqBPErSYAes88wSM_NEi6z-7-p-Q9magtURjPNOt4NZWLpO3o7TcmfniIY2cVyZr27F2qKCVwXZdiSk0E2ac4d8eLDxhUZU5vceTQYnuhBNmm06AewOfFuCmOlrjk Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: c8597a47523048e78c0791720c6e677a Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:47:31.758267 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:47:31.761197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c8597a47523048e78c0791720c6e677a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c8597a47523048e78c0791720c6e677a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c8597a47523048e78c0791720c6e677a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c8597a47523048e78c0791720c6e677a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:47:31.761197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.761197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: c8597a47523048e78c0791720c6e677a Aug 07 01:47:31.761197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.761197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-VolumesNegativeTest-1563497495-project Aug 07 01:47:31.761197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:47:31.761197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:47:31.761197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 3466f9e75a644b04869bf7f460b8642e Aug 07 01:47:31.761197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-VolumesNegativeTest-1563497495-project Aug 07 01:47:31.761197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Aug 07 01:47:31.761197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-65803002", "metadata": {"Type": "work"}}} Aug 07 01:47:31.761197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: Aug 07 01:47:31.761197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-65803002', 'metadata': {'Type': 'work'}}}} Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise error Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi '0' Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:47:31.762873 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 07 01:47:31.767313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 07 01:47:31.770389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 07 01:47:31.770389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.770389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-c6c823c1-7dc5-4c34-890f-ccda1a376b63 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:47:31.770389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-c6c823c1-7dc5-4c34-890f-ccda1a376b63 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.019595 Aug 07 01:47:31.770389 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1217/2165] 10.222.0.26 () {60 vars in 1156 bytes} [Sun Aug 7 01:47:31 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 22 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 07 01:47:31.778288 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6c49331c-65c8-458a-a37e-0bb1d58ec3b2 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Action: 'create', calling method: .version_select at 0x7f1fad14bca0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-221619564", "metadata": {"Type": "work"}}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6c49331c-65c8-458a-a37e-0bb1d58ec3b2 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Request method failure captured: Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 118 Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xmzGzN3tL5wLhfdjWHcU9TVXMQG8bLLObeSkPyWZKGoyxJrWuiqBPErSYAes88wSM_NEi6z-7-p-Q9magtURjPNOt4NZWLpO3o7TcmfniIY2cVyZr27F2qKCVwXZdiSk0E2ac4d8eLDxhUZU5vceTQYnuhBNmm06AewOfFuCmOlrjk Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: c8597a47523048e78c0791720c6e677a Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:47:31.785266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:47:31.787798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c8597a47523048e78c0791720c6e677a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c8597a47523048e78c0791720c6e677a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c8597a47523048e78c0791720c6e677a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c8597a47523048e78c0791720c6e677a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:47:31.787798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.787798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: c8597a47523048e78c0791720c6e677a Aug 07 01:47:31.787798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.787798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-VolumesNegativeTest-1563497495-project Aug 07 01:47:31.787798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:47:31.787798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:47:31.787798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: 3466f9e75a644b04869bf7f460b8642e Aug 07 01:47:31.787798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-VolumesNegativeTest-1563497495-project Aug 07 01:47:31.787798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Aug 07 01:47:31.787798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-221619564", "metadata": {"Type": "work"}}} Aug 07 01:47:31.787798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: Aug 07 01:47:31.787798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-221619564', 'metadata': {'Type': 'work'}}}} Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise error Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi '' Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:47:31.789821 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 07 01:47:31.792036 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 07 01:47:31.794572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 07 01:47:31.794572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.794572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6c49331c-65c8-458a-a37e-0bb1d58ec3b2 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:47:31.794572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6c49331c-65c8-458a-a37e-0bb1d58ec3b2 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.015455 Aug 07 01:47:31.794572 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 949/2166] 10.222.0.26 () {60 vars in 1156 bytes} [Sun Aug 7 01:47:31 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 17 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Aug 07 01:47:31.809173 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0c7713e5-8914-4e2a-86b5-722b91829fcb tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Calling method '.version_select at 0x7f1fb1bbcc10>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:47:31.856314 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-0c7713e5-8914-4e2a-86b5-722b91829fcb tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] HTTP exception thrown: Volume tempest-invalid-1041277589 could not be found. Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0c7713e5-8914-4e2a-86b5-722b91829fcb tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Request method failure captured: Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1041277589 HTTP/1.1 Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xmzGzN3tL5wLhfdjWHcU9TVXMQG8bLLObeSkPyWZKGoyxJrWuiqBPErSYAes88wSM_NEi6z-7-p-Q9magtURjPNOt4NZWLpO3o7TcmfniIY2cVyZr27F2qKCVwXZdiSk0E2ac4d8eLDxhUZU5vceTQYnuhBNmm06AewOfFuCmOlrjk Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: c8597a47523048e78c0791720c6e677a Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:47:31.860818 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:47:31.863189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c8597a47523048e78c0791720c6e677a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c8597a47523048e78c0791720c6e677a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c8597a47523048e78c0791720c6e677a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c8597a47523048e78c0791720c6e677a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:47:31.863189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.863189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: c8597a47523048e78c0791720c6e677a Aug 07 01:47:31.863189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.863189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-VolumesNegativeTest-1563497495-project Aug 07 01:47:31.863189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:47:31.863189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:47:31.863189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 3466f9e75a644b04869bf7f460b8642e Aug 07 01:47:31.863189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-VolumesNegativeTest-1563497495-project Aug 07 01:47:31.863189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: Aug 07 01:47:31.863189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Volume tempest-invalid-1041277589 could not be found. Aug 07 01:47:31.863189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': 'tempest-invalid-1041277589'} Aug 07 01:47:31.863189 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1041277589 could not be found. (HTTP 404) (Request-ID: req-cf54ccac-6c51-43af-b320-590c50d6f52e) Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:31.864568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 07 01:47:31.866465 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1041277589 could not be found. Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:47:31.868466 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1041277589 could not be found. Aug 07 01:47:31.870598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.870598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:31.870598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.870598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.870598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:31.870598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:31.870598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:47:31.870598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:47:31.870598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1041277589 could not be found. Aug 07 01:47:31.870598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.870598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0c7713e5-8914-4e2a-86b5-722b91829fcb tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Returning 404 to user: Volume tempest-invalid-1041277589 could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:47:31.870598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0c7713e5-8914-4e2a-86b5-722b91829fcb tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1041277589" status: 404 len: 99 microversion: 2.1 time: 0.062592 Aug 07 01:47:31.870598 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1218/2167] 10.222.0.26 () {58 vars in 1218 bytes} [Sun Aug 7 01:47:31 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1041277589 => generated 99 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Aug 07 01:47:31.882168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-50b39901-ca0f-4397-95d0-91846cb881f8 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.006756 Aug 07 01:47:31.887889 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 950/2168] 10.222.0.26 () {58 vars in 1140 bytes} [Sun Aug 7 01:47:31 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 14 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Aug 07 01:47:31.904204 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-3b2d4555-d2d6-4dc2-b2c9-baaafacc3def tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.006868 Aug 07 01:47:31.905257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1219/2169] 10.222.0.26 () {58 vars in 1137 bytes} [Sun Aug 7 01:47:31 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 9 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Aug 07 01:47:31.918334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ae2521e4-70d6-48b5-bb9d-83ed58e51421 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Calling method '.version_select at 0x7f1fad47cd30>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:47:31.986998 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-ae2521e4-70d6-48b5-bb9d-83ed58e51421 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] HTTP exception thrown: Volume 3844f93a-8ca8-4151-8317-ebe562c47b61 could not be found. Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ae2521e4-70d6-48b5-bb9d-83ed58e51421 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Request method failure captured: Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: DELETE /compute/v2.1/os-volumes/3844f93a-8ca8-4151-8317-ebe562c47b61 HTTP/1.1 Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xmzGzN3tL5wLhfdjWHcU9TVXMQG8bLLObeSkPyWZKGoyxJrWuiqBPErSYAes88wSM_NEi6z-7-p-Q9magtURjPNOt4NZWLpO3o7TcmfniIY2cVyZr27F2qKCVwXZdiSk0E2ac4d8eLDxhUZU5vceTQYnuhBNmm06AewOfFuCmOlrjk Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: c8597a47523048e78c0791720c6e677a Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:47:31.989619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:47:31.991441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c8597a47523048e78c0791720c6e677a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c8597a47523048e78c0791720c6e677a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c8597a47523048e78c0791720c6e677a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c8597a47523048e78c0791720c6e677a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:47:31.991441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.991441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: c8597a47523048e78c0791720c6e677a Aug 07 01:47:31.991441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:31.991441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-VolumesNegativeTest-1563497495-project Aug 07 01:47:31.991441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:47:31.991441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:47:31.991441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: 3466f9e75a644b04869bf7f460b8642e Aug 07 01:47:31.991441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-VolumesNegativeTest-1563497495-project Aug 07 01:47:31.991441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: Aug 07 01:47:31.991441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Volume 3844f93a-8ca8-4151-8317-ebe562c47b61 could not be found. Aug 07 01:47:31.991441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '3844f93a-8ca8-4151-8317-ebe562c47b61'} Aug 07 01:47:31.991441 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 3844f93a-8ca8-4151-8317-ebe562c47b61 could not be found. (HTTP 404) (Request-ID: req-acb6f162-0190-42b2-a195-a1a10f64d692) Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:31.992512 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 07 01:47:31.993932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 3844f93a-8ca8-4151-8317-ebe562c47b61 could not be found. Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 07 01:47:31.995857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:47:32.001225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 3844f93a-8ca8-4151-8317-ebe562c47b61 could not be found. Aug 07 01:47:32.001225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:32.001225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:32.001225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:32.001225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:32.001225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:32.001225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:32.001225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:47:32.001225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:47:32.001225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 3844f93a-8ca8-4151-8317-ebe562c47b61 could not be found. Aug 07 01:47:32.001225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:47:32.001225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ae2521e4-70d6-48b5-bb9d-83ed58e51421 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Returning 404 to user: Volume 3844f93a-8ca8-4151-8317-ebe562c47b61 could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:47:32.001225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ae2521e4-70d6-48b5-bb9d-83ed58e51421 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/3844f93a-8ca8-4151-8317-ebe562c47b61" status: 404 len: 109 microversion: 2.1 time: 0.077953 Aug 07 01:47:32.001225 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 951/2170] 10.222.0.26 () {58 vars in 1248 bytes} [Sun Aug 7 01:47:31 2022] DELETE /compute/v2.1/os-volumes/3844f93a-8ca8-4151-8317-ebe562c47b61 => generated 109 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 07 01:47:32.008085 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-84d554df-8523-4bec-8fc6-cf2fd28f0790 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Calling method '.version_select at 0x7f1fad20af70>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:47:32.056307 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-84d554df-8523-4bec-8fc6-cf2fd28f0790 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] HTTP exception thrown: Volume 13b2af0a-d1be-47c1-abcf-744ec8ca80d8 could not be found. Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-84d554df-8523-4bec-8fc6-cf2fd28f0790 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Request method failure captured: Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/os-volumes/13b2af0a-d1be-47c1-abcf-744ec8ca80d8 HTTP/1.1 Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xmzGzN3tL5wLhfdjWHcU9TVXMQG8bLLObeSkPyWZKGoyxJrWuiqBPErSYAes88wSM_NEi6z-7-p-Q9magtURjPNOt4NZWLpO3o7TcmfniIY2cVyZr27F2qKCVwXZdiSk0E2ac4d8eLDxhUZU5vceTQYnuhBNmm06AewOfFuCmOlrjk Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: c8597a47523048e78c0791720c6e677a Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:47:32.059364 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:47:32.061667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c8597a47523048e78c0791720c6e677a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c8597a47523048e78c0791720c6e677a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c8597a47523048e78c0791720c6e677a"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c8597a47523048e78c0791720c6e677a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:47:32.061667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:32.061667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: c8597a47523048e78c0791720c6e677a Aug 07 01:47:32.061667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-VolumesNegativeTest-1563497495 Aug 07 01:47:32.061667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-VolumesNegativeTest-1563497495-project Aug 07 01:47:32.061667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:47:32.061667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:47:32.061667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: 3466f9e75a644b04869bf7f460b8642e Aug 07 01:47:32.061667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-VolumesNegativeTest-1563497495-project Aug 07 01:47:32.061667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: Aug 07 01:47:32.061667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Volume 13b2af0a-d1be-47c1-abcf-744ec8ca80d8 could not be found. Aug 07 01:47:32.061667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '13b2af0a-d1be-47c1-abcf-744ec8ca80d8'} Aug 07 01:47:32.061667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 13b2af0a-d1be-47c1-abcf-744ec8ca80d8 could not be found. (HTTP 404) (Request-ID: req-3dbfd4d1-6f1f-4064-9328-985fe28bcdd8) Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:32.062898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 07 01:47:32.064647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 13b2af0a-d1be-47c1-abcf-744ec8ca80d8 could not be found. Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 07 01:47:32.066410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:47:32.068167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 13b2af0a-d1be-47c1-abcf-744ec8ca80d8 could not be found. Aug 07 01:47:32.068167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:32.068167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:47:32.068167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:32.068167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:47:32.068167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:47:32.068167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:47:32.068167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:47:32.068167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:47:32.068167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 13b2af0a-d1be-47c1-abcf-744ec8ca80d8 could not be found. Aug 07 01:47:32.068167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:47:32.068167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-84d554df-8523-4bec-8fc6-cf2fd28f0790 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] Returning 404 to user: Volume 13b2af0a-d1be-47c1-abcf-744ec8ca80d8 could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:47:32.068167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-84d554df-8523-4bec-8fc6-cf2fd28f0790 tempest-VolumesNegativeTest-1563497495 tempest-VolumesNegativeTest-1563497495-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/13b2af0a-d1be-47c1-abcf-744ec8ca80d8" status: 404 len: 109 microversion: 2.1 time: 0.060055 Aug 07 01:47:32.068167 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1220/2171] 10.222.0.26 () {58 vars in 1245 bytes} [Sun Aug 7 01:47:31 2022] GET /compute/v2.1/os-volumes/13b2af0a-d1be-47c1-abcf-744ec8ca80d8 => generated 109 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:48:24.971275 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=6 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:48:24.973663 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=6 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:48:25.340822 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-47e084be-873d-47cb-a6fb-03f89311b67d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Action: 'create', calling method: .version_select at 0x7f1fad398ca0>, body: {"keypair": {"name": "tempest-TestShelveInstance-1538091592", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL9xbh9lhyOSpnnoCjfHlSLn2glRpYN72AiM1S+LwGYgz6cEHMK/bG++iOEzvhh/CVDT5hvs2g4IrTHYsBcBwQ72Mpiu4wO50uNRnZvFnFMzGYPvQMH0cL17SSTNE8dJQA=="}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:48:25.359958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-47e084be-873d-47cb-a6fb-03f89311b67d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Getting quotas for project a3df9bc7e5f54e938a0ab2bdc841ecd3. Resources: {'key_pairs'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:48:25.365124 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-47e084be-873d-47cb-a6fb-03f89311b67d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Getting quotas for user 3059ad20d01b49adbb6dbb8ca75e451c and project a3df9bc7e5f54e938a0ab2bdc841ecd3. Resources: {'key_pairs'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:48:25.380659 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR oslo.messaging._drivers.impl_rabbit [None req-47e084be-873d-47cb-a6fb-03f89311b67d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [69b55aee-c31b-4a95-8518-f1d33bb07651] AMQP server on 10.222.0.26:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Aug 07 01:48:26.390152 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [None req-47e084be-873d-47cb-a6fb-03f89311b67d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [69b55aee-c31b-4a95-8518-f1d33bb07651] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 43526. Aug 07 01:48:26.419878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-47e084be-873d-47cb-a6fb-03f89311b67d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 1.463207 Aug 07 01:48:26.422127 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 952/2172] 10.222.0.26 () {60 vars in 1159 bytes} [Sun Aug 7 01:48:24 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 1474 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:48:30.024568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=6 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:48:30.025813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=6 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:48:30.036033 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-829aa4e2-72d9-4297-b577-8a758e4ac458 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Action: 'create', calling method: .version_select at 0x7f1fad7358b0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-115664317", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFXKuy8yjR0nc5QjZAgnC7a4GCBG27iPrXM+FJYFGU+gTR+6H1m7pBm4Xmcwtu6CoIuPARmgG0PsbK+ldtINW8E9Z/swbfI1YKLKECuV8M0J94hOLBTKTpzSeVnNlPBezg=="}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:48:30.070183 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-829aa4e2-72d9-4297-b577-8a758e4ac458 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Getting quotas for project 673afe3c625d419e89fe80f3db4574fb. Resources: {'key_pairs'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:48:30.080844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-829aa4e2-72d9-4297-b577-8a758e4ac458 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Getting quotas for user a11fb6edaae94f8ba2214e5ec0c65972 and project 673afe3c625d419e89fe80f3db4574fb. Resources: {'key_pairs'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:48:30.107902 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR oslo.messaging._drivers.impl_rabbit [None req-829aa4e2-72d9-4297-b577-8a758e4ac458 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [eafb0ecb-6d84-44f7-80a1-140e7b7b2f53] AMQP server on 10.222.0.26:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Aug 07 01:48:31.122138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [None req-829aa4e2-72d9-4297-b577-8a758e4ac458 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [eafb0ecb-6d84-44f7-80a1-140e7b7b2f53] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 43664. Aug 07 01:48:31.146209 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-829aa4e2-72d9-4297-b577-8a758e4ac458 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 1.124835 Aug 07 01:48:31.159001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1221/2173] 10.222.0.26 () {60 vars in 1159 bytes} [Sun Aug 7 01:48:30 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 1138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:48:31.355286 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-440376613", "imageRef": "a21f4438-12bc-4d11-a781-6ed28670ea00", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-115664317", "networks": [{"uuid": "ed934e06-e878-44ca-b0cd-4ffe32e81e1a"}]}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:48:31.476958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.network.neutron [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] validate_networks() for [('ed934e06-e878-44ca-b0cd-4ffe32e81e1a', None, None, None, None, None)] {{(pid=117984) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:48:32.353767 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Flavor limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:48:32.354596 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Image limits 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:48:32.355548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Flavor pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:48:32.356312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Image pref 0:0:0 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:48:32.357111 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.virt.hardware [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117984) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:48:32.382877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:32.383725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:32.384623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:32.386053 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:32.387701 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:32.388637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:32.428668 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Getting quotas for project 673afe3c625d419e89fe80f3db4574fb. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:48:32.433910 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.quota [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Getting quotas for user a11fb6edaae94f8ba2214e5ec0c65972 and project 673afe3c625d419e89fe80f3db4574fb. Resources: {'ram', 'cores', 'instances'} {{(pid=117984) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:48:32.449405 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Going to run 1 instances... {{(pid=117984) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:48:32.453309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a21f4438-12bc-4d11-a781-6ed28670ea00',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117984) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:48:32.524816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-80490d1a-9283-48f5-92b2-66cb7a2b5df9 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.178024 Aug 07 01:48:32.526282 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 953/2174] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:48:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 1181 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:48:32.540479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-35d9a513-f623-453d-8a28-378e1c1b1303 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:32.542305 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-35d9a513-f623-453d-8a28-378e1c1b1303 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:32.676428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-35d9a513-f623-453d-8a28-378e1c1b1303 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:32.698917 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-35d9a513-f623-453d-8a28-378e1c1b1303 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1257 microversion: 2.1 time: 0.158212 Aug 07 01:48:32.700647 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1222/2175] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:32 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1257 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:33.730224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e07cda1e-3b67-4e8b-9b39-4f85af9c0c1e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:33.731872 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e07cda1e-3b67-4e8b-9b39-4f85af9c0c1e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:33.741394 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e07cda1e-3b67-4e8b-9b39-4f85af9c0c1e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:33.742194 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e07cda1e-3b67-4e8b-9b39-4f85af9c0c1e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:33.743082 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e07cda1e-3b67-4e8b-9b39-4f85af9c0c1e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:33.744839 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e07cda1e-3b67-4e8b-9b39-4f85af9c0c1e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:33.745661 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e07cda1e-3b67-4e8b-9b39-4f85af9c0c1e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:33.746345 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e07cda1e-3b67-4e8b-9b39-4f85af9c0c1e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:33.876752 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e07cda1e-3b67-4e8b-9b39-4f85af9c0c1e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:33.885426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e07cda1e-3b67-4e8b-9b39-4f85af9c0c1e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1253 microversion: 2.1 time: 0.163555 Aug 07 01:48:33.886121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 954/2176] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:33 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1253 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:34.922646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-51dc6318-316c-4ad9-a77e-7d27e085a8ac tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:34.925980 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-51dc6318-316c-4ad9-a77e-7d27e085a8ac tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:34.945533 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51dc6318-316c-4ad9-a77e-7d27e085a8ac tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:34.947291 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51dc6318-316c-4ad9-a77e-7d27e085a8ac tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:34.948040 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51dc6318-316c-4ad9-a77e-7d27e085a8ac tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:34.953349 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51dc6318-316c-4ad9-a77e-7d27e085a8ac tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:34.954885 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51dc6318-316c-4ad9-a77e-7d27e085a8ac tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:34.956491 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-51dc6318-316c-4ad9-a77e-7d27e085a8ac tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:35.112813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-51dc6318-316c-4ad9-a77e-7d27e085a8ac tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:35.126674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-51dc6318-316c-4ad9-a77e-7d27e085a8ac tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1315 microversion: 2.1 time: 0.217699 Aug 07 01:48:35.127275 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1223/2177] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:34 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1315 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:36.178442 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9ede4b11-ad61-421b-b670-e5ec9653752b tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:36.179748 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9ede4b11-ad61-421b-b670-e5ec9653752b tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:36.198291 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ede4b11-ad61-421b-b670-e5ec9653752b tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:36.199459 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ede4b11-ad61-421b-b670-e5ec9653752b tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:36.200466 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ede4b11-ad61-421b-b670-e5ec9653752b tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:36.202470 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ede4b11-ad61-421b-b670-e5ec9653752b tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:36.203400 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ede4b11-ad61-421b-b670-e5ec9653752b tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:36.204099 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9ede4b11-ad61-421b-b670-e5ec9653752b tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:36.708284 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9ede4b11-ad61-421b-b670-e5ec9653752b tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:36.722987 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9ede4b11-ad61-421b-b670-e5ec9653752b tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1357 microversion: 2.1 time: 0.549926 Aug 07 01:48:36.734943 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 955/2178] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:36 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1357 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:37.749291 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d0d2a422-e031-4cab-bb10-c384afb6f667 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:37.750558 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d0d2a422-e031-4cab-bb10-c384afb6f667 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:37.763594 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0d2a422-e031-4cab-bb10-c384afb6f667 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:37.768728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0d2a422-e031-4cab-bb10-c384afb6f667 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:37.769438 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0d2a422-e031-4cab-bb10-c384afb6f667 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:37.770365 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0d2a422-e031-4cab-bb10-c384afb6f667 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:37.770996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0d2a422-e031-4cab-bb10-c384afb6f667 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:37.771600 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d0d2a422-e031-4cab-bb10-c384afb6f667 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:38.449252 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-de24a711-8e12-48fb-a25c-9c400aee7e38 req-10ab74b5-bba3-4328-a99d-3a01be166c96 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "65635777-b502-4848-8327-c39ccd195e5d", "tag": "7fe8e7c8-c5ab-41e4-9fdb-393c265c2479"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:48:38.473204 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-de24a711-8e12-48fb-a25c-9c400aee7e38 req-10ab74b5-bba3-4328-a99d-3a01be166c96 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:38.474156 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-de24a711-8e12-48fb-a25c-9c400aee7e38 req-10ab74b5-bba3-4328-a99d-3a01be166c96 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:38.478600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-de24a711-8e12-48fb-a25c-9c400aee7e38 req-10ab74b5-bba3-4328-a99d-3a01be166c96 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:38.500421 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-de24a711-8e12-48fb-a25c-9c400aee7e38 req-10ab74b5-bba3-4328-a99d-3a01be166c96 service nova] Creating event network-changed:7fe8e7c8-c5ab-41e4-9fdb-393c265c2479 for instance 65635777-b502-4848-8327-c39ccd195e5d on hv-cinder-maste Aug 07 01:48:38.507613 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-de24a711-8e12-48fb-a25c-9c400aee7e38 req-10ab74b5-bba3-4328-a99d-3a01be166c96 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062442 Aug 07 01:48:38.509311 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 956/2179] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:48:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:48:38.575562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d0d2a422-e031-4cab-bb10-c384afb6f667 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:38.587510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d0d2a422-e031-4cab-bb10-c384afb6f667 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1357 microversion: 2.1 time: 0.841959 Aug 07 01:48:38.605133 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1224/2180] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:37 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1357 bytes in 861 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:39.620048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-178eb980-ca2e-47d4-991d-5508fb742799 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:39.621651 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-178eb980-ca2e-47d4-991d-5508fb742799 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:39.629935 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-178eb980-ca2e-47d4-991d-5508fb742799 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:39.630865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-178eb980-ca2e-47d4-991d-5508fb742799 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:39.641064 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-178eb980-ca2e-47d4-991d-5508fb742799 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:39.642566 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-178eb980-ca2e-47d4-991d-5508fb742799 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:39.647087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-178eb980-ca2e-47d4-991d-5508fb742799 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:39.648032 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-178eb980-ca2e-47d4-991d-5508fb742799 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:39.901250 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-178eb980-ca2e-47d4-991d-5508fb742799 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:39.911448 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-178eb980-ca2e-47d4-991d-5508fb742799 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1357 microversion: 2.1 time: 0.295809 Aug 07 01:48:39.926978 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 957/2181] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:39 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1357 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:40.935545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5e1e0fbb-6ee4-442c-9027-5af332aadf25 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:40.937129 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5e1e0fbb-6ee4-442c-9027-5af332aadf25 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:40.948994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5e1e0fbb-6ee4-442c-9027-5af332aadf25 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:40.953936 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5e1e0fbb-6ee4-442c-9027-5af332aadf25 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:40.957348 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5e1e0fbb-6ee4-442c-9027-5af332aadf25 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:40.958530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5e1e0fbb-6ee4-442c-9027-5af332aadf25 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:40.960554 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5e1e0fbb-6ee4-442c-9027-5af332aadf25 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:40.961986 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5e1e0fbb-6ee4-442c-9027-5af332aadf25 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:41.216002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5e1e0fbb-6ee4-442c-9027-5af332aadf25 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:41.237400 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5e1e0fbb-6ee4-442c-9027-5af332aadf25 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1357 microversion: 2.1 time: 0.304792 Aug 07 01:48:41.239420 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1225/2182] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:40 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1357 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:42.263378 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-c058dc1f-4c7c-4447-a4e5-c0e4f70c697f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:42.273920 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-c058dc1f-4c7c-4447-a4e5-c0e4f70c697f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:42.299884 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c058dc1f-4c7c-4447-a4e5-c0e4f70c697f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:42.301270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c058dc1f-4c7c-4447-a4e5-c0e4f70c697f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:42.302309 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c058dc1f-4c7c-4447-a4e5-c0e4f70c697f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:42.304066 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c058dc1f-4c7c-4447-a4e5-c0e4f70c697f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:42.310507 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c058dc1f-4c7c-4447-a4e5-c0e4f70c697f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:42.313048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-c058dc1f-4c7c-4447-a4e5-c0e4f70c697f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:42.415177 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:48:42.448385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=10 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:48:42.578966 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-c058dc1f-4c7c-4447-a4e5-c0e4f70c697f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:42.591151 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-c058dc1f-4c7c-4447-a4e5-c0e4f70c697f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1357 microversion: 2.1 time: 0.334264 Aug 07 01:48:42.593266 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 958/2183] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:42 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1357 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:48:43.627943 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a9c1aab3-a136-48a0-8d56-d32b888f65bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:43.629020 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a9c1aab3-a136-48a0-8d56-d32b888f65bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:43.632555 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=6 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:48:43.644661 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9c1aab3-a136-48a0-8d56-d32b888f65bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:43.645860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9c1aab3-a136-48a0-8d56-d32b888f65bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:43.647253 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9c1aab3-a136-48a0-8d56-d32b888f65bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:43.648832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9c1aab3-a136-48a0-8d56-d32b888f65bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:43.649919 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9c1aab3-a136-48a0-8d56-d32b888f65bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:43.651307 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a9c1aab3-a136-48a0-8d56-d32b888f65bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:43.929874 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a9c1aab3-a136-48a0-8d56-d32b888f65bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:43.942541 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a9c1aab3-a136-48a0-8d56-d32b888f65bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1357 microversion: 2.1 time: 0.321586 Aug 07 01:48:43.943578 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1226/2184] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:43 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1357 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:44.578866 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:48:44.579207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:48:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:48:44.977098 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-d8cb79f3-dbd6-45d7-a0f3-f0c3e4134ea1 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:44.978084 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-d8cb79f3-dbd6-45d7-a0f3-f0c3e4134ea1 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:44.986108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8cb79f3-dbd6-45d7-a0f3-f0c3e4134ea1 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:44.986571 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8cb79f3-dbd6-45d7-a0f3-f0c3e4134ea1 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:44.987022 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8cb79f3-dbd6-45d7-a0f3-f0c3e4134ea1 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:44.987818 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8cb79f3-dbd6-45d7-a0f3-f0c3e4134ea1 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:44.988458 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8cb79f3-dbd6-45d7-a0f3-f0c3e4134ea1 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:44.988931 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-d8cb79f3-dbd6-45d7-a0f3-f0c3e4134ea1 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:45.145078 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-d8cb79f3-dbd6-45d7-a0f3-f0c3e4134ea1 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:45.153798 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-d8cb79f3-dbd6-45d7-a0f3-f0c3e4134ea1 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1357 microversion: 2.1 time: 0.181061 Aug 07 01:48:45.154386 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 959/2185] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:44 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1357 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:48:45.952376 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1467205245", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1538091592", "security_groups": [{"name": "tempest-secgroup-smoke-1951819544"}], "networks": [{"uuid": "384bcd20-2ca4-43e0-9733-babcc24354c3"}], "block_device_mapping_v2": [{"uuid": "d70ce84f-81fc-4270-b086-9c968c67636e", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:48:46.140745 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.network.neutron [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] validate_networks() for [('384bcd20-2ca4-43e0-9733-babcc24354c3', None, None, None, None, None)] {{(pid=117983) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 07 01:48:46.177891 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-69c41f3c-35bd-4afb-be0c-2a1e76bc9517 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:46.178629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-69c41f3c-35bd-4afb-be0c-2a1e76bc9517 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:46.186851 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69c41f3c-35bd-4afb-be0c-2a1e76bc9517 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:46.187473 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69c41f3c-35bd-4afb-be0c-2a1e76bc9517 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:46.187979 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69c41f3c-35bd-4afb-be0c-2a1e76bc9517 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:46.188848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69c41f3c-35bd-4afb-be0c-2a1e76bc9517 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:46.189381 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69c41f3c-35bd-4afb-be0c-2a1e76bc9517 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:46.189867 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-69c41f3c-35bd-4afb-be0c-2a1e76bc9517 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:46.370100 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-69c41f3c-35bd-4afb-be0c-2a1e76bc9517 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:46.380829 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-69c41f3c-35bd-4afb-be0c-2a1e76bc9517 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1357 microversion: 2.1 time: 0.206357 Aug 07 01:48:46.381630 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 960/2186] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:46 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1357 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:48:46.903906 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Flavor limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 07 01:48:46.904359 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Image limits 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 07 01:48:46.904898 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Flavor pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 07 01:48:46.905798 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Image pref 0:0:0 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 07 01:48:46.906207 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.virt.hardware [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117983) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 07 01:48:46.915866 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:46.916385 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:46.916827 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:46.917566 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:46.918020 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:46.918447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:46.937595 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Getting quotas for project a3df9bc7e5f54e938a0ab2bdc841ecd3. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 07 01:48:46.942531 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.quota [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Getting quotas for user 3059ad20d01b49adbb6dbb8ca75e451c and project a3df9bc7e5f54e938a0ab2bdc841ecd3. Resources: {'ram', 'cores', 'instances'} {{(pid=117983) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 07 01:48:47.026057 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Going to run 1 instances... {{(pid=117983) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Aug 07 01:48:47.028996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='d70ce84f-81fc-4270-b086-9c968c67636e',volume_size=None,volume_type=None)] {{(pid=117983) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Aug 07 01:48:47.193079 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-438f44f6-5b73-4d32-b0e5-fea915e312f6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.246821 Aug 07 01:48:47.200347 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1227/2187] 10.222.0.26 () {60 vars in 1147 bytes} [Sun Aug 7 01:48:45 2022] POST /compute/v2.1/servers => generated 406 bytes in 1250 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 07 01:48:47.209905 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-5e88e866-3972-4bfd-be5a-55a490d00e92 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:47.211125 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-5e88e866-3972-4bfd-be5a-55a490d00e92 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:47.315794 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-5e88e866-3972-4bfd-be5a-55a490d00e92 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:47.327899 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-5e88e866-3972-4bfd-be5a-55a490d00e92 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1089 microversion: 2.1 time: 0.124580 Aug 07 01:48:47.329015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 961/2188] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:47 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1089 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:47.407840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-755f56f4-48b9-42ec-b556-08f627491389 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:47.408937 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-755f56f4-48b9-42ec-b556-08f627491389 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:47.415801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-755f56f4-48b9-42ec-b556-08f627491389 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:47.416337 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-755f56f4-48b9-42ec-b556-08f627491389 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:47.416771 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-755f56f4-48b9-42ec-b556-08f627491389 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:47.417712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-755f56f4-48b9-42ec-b556-08f627491389 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:47.418383 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-755f56f4-48b9-42ec-b556-08f627491389 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:47.418850 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-755f56f4-48b9-42ec-b556-08f627491389 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:47.657019 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-755f56f4-48b9-42ec-b556-08f627491389 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:47.672857 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-755f56f4-48b9-42ec-b556-08f627491389 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1357 microversion: 2.1 time: 0.268994 Aug 07 01:48:47.674294 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1228/2189] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:47 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1357 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:48:48.350548 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-685d1854-f1f8-42b5-9899-ab7493b137e8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:48.351210 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-685d1854-f1f8-42b5-9899-ab7493b137e8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:48.360722 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-685d1854-f1f8-42b5-9899-ab7493b137e8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:48.361271 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-685d1854-f1f8-42b5-9899-ab7493b137e8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:48.361744 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-685d1854-f1f8-42b5-9899-ab7493b137e8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:48.362577 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-685d1854-f1f8-42b5-9899-ab7493b137e8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:48.363059 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-685d1854-f1f8-42b5-9899-ab7493b137e8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:48.363494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-685d1854-f1f8-42b5-9899-ab7493b137e8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:48.458991 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-685d1854-f1f8-42b5-9899-ab7493b137e8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:48.475332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-685d1854-f1f8-42b5-9899-ab7493b137e8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1139 microversion: 2.1 time: 0.129062 Aug 07 01:48:48.477116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 962/2190] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:48 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1139 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:48:48.715816 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a14c18ac-6a5d-416e-85a1-09a53004af2a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:48.718356 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a14c18ac-6a5d-416e-85a1-09a53004af2a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:48.733923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a14c18ac-6a5d-416e-85a1-09a53004af2a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:48.734994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a14c18ac-6a5d-416e-85a1-09a53004af2a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:48.735830 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a14c18ac-6a5d-416e-85a1-09a53004af2a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:48.737365 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a14c18ac-6a5d-416e-85a1-09a53004af2a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:48.738417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a14c18ac-6a5d-416e-85a1-09a53004af2a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:48.742458 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a14c18ac-6a5d-416e-85a1-09a53004af2a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:49.028690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a14c18ac-6a5d-416e-85a1-09a53004af2a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:49.040004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a14c18ac-6a5d-416e-85a1-09a53004af2a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1357 microversion: 2.1 time: 0.332811 Aug 07 01:48:49.041328 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1229/2191] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:48 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1357 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:49.518440 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-303ef592-35ea-4273-a427-7ee4cc0ebab6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:49.519897 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-303ef592-35ea-4273-a427-7ee4cc0ebab6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:49.539713 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-303ef592-35ea-4273-a427-7ee4cc0ebab6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:49.540953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-303ef592-35ea-4273-a427-7ee4cc0ebab6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:49.542752 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-303ef592-35ea-4273-a427-7ee4cc0ebab6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:49.545104 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-303ef592-35ea-4273-a427-7ee4cc0ebab6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:49.546709 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-303ef592-35ea-4273-a427-7ee4cc0ebab6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:49.547489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-303ef592-35ea-4273-a427-7ee4cc0ebab6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:49.676961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-303ef592-35ea-4273-a427-7ee4cc0ebab6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:49.695079 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-303ef592-35ea-4273-a427-7ee4cc0ebab6 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1195 microversion: 2.1 time: 0.185432 Aug 07 01:48:49.703678 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 963/2192] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:49 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1195 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:50.074726 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f4467175-1a12-4d23-8d90-c386d96855e3 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:50.075856 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f4467175-1a12-4d23-8d90-c386d96855e3 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:50.089838 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4467175-1a12-4d23-8d90-c386d96855e3 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:50.090727 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4467175-1a12-4d23-8d90-c386d96855e3 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:50.091549 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4467175-1a12-4d23-8d90-c386d96855e3 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:50.093755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4467175-1a12-4d23-8d90-c386d96855e3 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:50.095160 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4467175-1a12-4d23-8d90-c386d96855e3 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:50.096049 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f4467175-1a12-4d23-8d90-c386d96855e3 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:50.372840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f4467175-1a12-4d23-8d90-c386d96855e3 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:50.397306 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f4467175-1a12-4d23-8d90-c386d96855e3 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1357 microversion: 2.1 time: 0.326613 Aug 07 01:48:50.399674 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1230/2193] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:50 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1357 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:50.744579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f0a6a872-4229-40c6-bde0-a8e197819f5c tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:50.746959 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f0a6a872-4229-40c6-bde0-a8e197819f5c tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:50.762491 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0a6a872-4229-40c6-bde0-a8e197819f5c tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:50.765903 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0a6a872-4229-40c6-bde0-a8e197819f5c tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:50.768160 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0a6a872-4229-40c6-bde0-a8e197819f5c tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:50.769686 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0a6a872-4229-40c6-bde0-a8e197819f5c tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:50.770636 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0a6a872-4229-40c6-bde0-a8e197819f5c tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:50.771515 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f0a6a872-4229-40c6-bde0-a8e197819f5c tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:51.076368 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-86b8c16c-a8e0-4868-a309-0c30c7b959c4 req-94147649-7946-497e-99a5-6a6a6a294f6d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "65635777-b502-4848-8327-c39ccd195e5d", "name": "network-vif-plugged", "status": "completed", "tag": "7fe8e7c8-c5ab-41e4-9fdb-393c265c2479"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:48:51.098188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-86b8c16c-a8e0-4868-a309-0c30c7b959c4 req-94147649-7946-497e-99a5-6a6a6a294f6d service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:51.098685 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-86b8c16c-a8e0-4868-a309-0c30c7b959c4 req-94147649-7946-497e-99a5-6a6a6a294f6d service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:51.099457 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-86b8c16c-a8e0-4868-a309-0c30c7b959c4 req-94147649-7946-497e-99a5-6a6a6a294f6d service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:51.115536 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-86b8c16c-a8e0-4868-a309-0c30c7b959c4 req-94147649-7946-497e-99a5-6a6a6a294f6d service nova] Creating event network-vif-plugged:7fe8e7c8-c5ab-41e4-9fdb-393c265c2479 for instance 65635777-b502-4848-8327-c39ccd195e5d on hv-cinder-maste Aug 07 01:48:51.121628 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR oslo.messaging._drivers.impl_rabbit [req-86b8c16c-a8e0-4868-a309-0c30c7b959c4 req-94147649-7946-497e-99a5-6a6a6a294f6d service nova] [bf6a8231-c09d-4642-ae32-3e7bbcbccca0] AMQP server on 10.222.0.26:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Aug 07 01:48:51.220427 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f0a6a872-4229-40c6-bde0-a8e197819f5c tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:51.233974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f0a6a872-4229-40c6-bde0-a8e197819f5c tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1261 microversion: 2.1 time: 0.497421 Aug 07 01:48:51.235091 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 964/2194] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:50 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1261 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:51.434375 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a5f9d6dc-a1b8-439d-b2f1-1657629d512c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:51.434983 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a5f9d6dc-a1b8-439d-b2f1-1657629d512c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:51.442833 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5f9d6dc-a1b8-439d-b2f1-1657629d512c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:51.443326 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5f9d6dc-a1b8-439d-b2f1-1657629d512c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:51.443961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5f9d6dc-a1b8-439d-b2f1-1657629d512c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:51.444757 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5f9d6dc-a1b8-439d-b2f1-1657629d512c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:51.445187 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5f9d6dc-a1b8-439d-b2f1-1657629d512c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:51.445809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5f9d6dc-a1b8-439d-b2f1-1657629d512c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:52.039538 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-a5f9d6dc-a1b8-439d-b2f1-1657629d512c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:52.053017 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a5f9d6dc-a1b8-439d-b2f1-1657629d512c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1357 microversion: 2.1 time: 0.622115 Aug 07 01:48:52.053932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 965/2195] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:51 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1357 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:52.131658 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [req-86b8c16c-a8e0-4868-a309-0c30c7b959c4 req-94147649-7946-497e-99a5-6a6a6a294f6d service nova] [bf6a8231-c09d-4642-ae32-3e7bbcbccca0] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 44060. Aug 07 01:48:52.138103 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-86b8c16c-a8e0-4868-a309-0c30c7b959c4 req-94147649-7946-497e-99a5-6a6a6a294f6d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 1.368560 Aug 07 01:48:52.139263 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1231/2196] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:48:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 1371 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:48:52.272333 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-00bd8199-e8c6-4a2e-8272-fb203748eeba tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:52.273458 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-00bd8199-e8c6-4a2e-8272-fb203748eeba tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:52.286628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00bd8199-e8c6-4a2e-8272-fb203748eeba tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:52.287777 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00bd8199-e8c6-4a2e-8272-fb203748eeba tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:52.288756 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00bd8199-e8c6-4a2e-8272-fb203748eeba tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:52.290223 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00bd8199-e8c6-4a2e-8272-fb203748eeba tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:52.291506 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00bd8199-e8c6-4a2e-8272-fb203748eeba tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:52.292283 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-00bd8199-e8c6-4a2e-8272-fb203748eeba tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:52.523720 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-b2719f78-2007-485f-a9ac-10e461102af8 req-5ff25d33-7ac0-41b7-9133-ccf4203d8c54 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "768918d9-86a2-4c55-ba2a-57f5c1dcd234", "tag": "ebb91259-4e43-449e-b5bb-c646943874ee"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:48:52.543246 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-b2719f78-2007-485f-a9ac-10e461102af8 req-5ff25d33-7ac0-41b7-9133-ccf4203d8c54 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:52.544367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-b2719f78-2007-485f-a9ac-10e461102af8 req-5ff25d33-7ac0-41b7-9133-ccf4203d8c54 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:52.545170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-b2719f78-2007-485f-a9ac-10e461102af8 req-5ff25d33-7ac0-41b7-9133-ccf4203d8c54 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:52.568613 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-b2719f78-2007-485f-a9ac-10e461102af8 req-5ff25d33-7ac0-41b7-9133-ccf4203d8c54 service nova] Creating event network-changed:ebb91259-4e43-449e-b5bb-c646943874ee for instance 768918d9-86a2-4c55-ba2a-57f5c1dcd234 on hv-cinder-maste Aug 07 01:48:52.577984 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-b2719f78-2007-485f-a9ac-10e461102af8 req-5ff25d33-7ac0-41b7-9133-ccf4203d8c54 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062160 Aug 07 01:48:52.579051 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1232/2197] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:48:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:48:52.591805 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-00bd8199-e8c6-4a2e-8272-fb203748eeba tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:52.602453 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-00bd8199-e8c6-4a2e-8272-fb203748eeba tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1261 microversion: 2.1 time: 0.338508 Aug 07 01:48:52.602970 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 966/2198] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:52 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1261 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:53.078649 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7d4e0b12-2938-461f-a893-c537043b89fa tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:53.079420 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7d4e0b12-2938-461f-a893-c537043b89fa tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:53.086659 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d4e0b12-2938-461f-a893-c537043b89fa tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:53.087218 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d4e0b12-2938-461f-a893-c537043b89fa tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:53.087735 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d4e0b12-2938-461f-a893-c537043b89fa tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:53.088564 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d4e0b12-2938-461f-a893-c537043b89fa tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:53.089071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d4e0b12-2938-461f-a893-c537043b89fa tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:53.089545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7d4e0b12-2938-461f-a893-c537043b89fa tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:53.271940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7d4e0b12-2938-461f-a893-c537043b89fa tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:53.289968 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7d4e0b12-2938-461f-a893-c537043b89fa tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1357 microversion: 2.1 time: 0.216155 Aug 07 01:48:53.290569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1233/2199] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:53 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1357 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:53.636456 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-732909d9-01e8-4a48-93a4-203d093e5eda tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:53.637270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-732909d9-01e8-4a48-93a4-203d093e5eda tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:53.644729 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-732909d9-01e8-4a48-93a4-203d093e5eda tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:53.645232 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-732909d9-01e8-4a48-93a4-203d093e5eda tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:53.645681 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-732909d9-01e8-4a48-93a4-203d093e5eda tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:53.646444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-732909d9-01e8-4a48-93a4-203d093e5eda tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:53.647052 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-732909d9-01e8-4a48-93a4-203d093e5eda tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:53.647520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-732909d9-01e8-4a48-93a4-203d093e5eda tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:53.805979 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-732909d9-01e8-4a48-93a4-203d093e5eda tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:53.814426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-732909d9-01e8-4a48-93a4-203d093e5eda tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1261 microversion: 2.1 time: 0.182261 Aug 07 01:48:53.815120 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 967/2200] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:53 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1261 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:54.319801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e4816db7-3dbd-472b-9ff0-4f800707bb48 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:54.320860 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e4816db7-3dbd-472b-9ff0-4f800707bb48 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:54.330529 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4816db7-3dbd-472b-9ff0-4f800707bb48 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:54.331354 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4816db7-3dbd-472b-9ff0-4f800707bb48 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:54.332310 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4816db7-3dbd-472b-9ff0-4f800707bb48 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:54.333524 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4816db7-3dbd-472b-9ff0-4f800707bb48 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:54.334554 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4816db7-3dbd-472b-9ff0-4f800707bb48 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:54.335309 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e4816db7-3dbd-472b-9ff0-4f800707bb48 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:54.549654 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e4816db7-3dbd-472b-9ff0-4f800707bb48 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:54.565033 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e4816db7-3dbd-472b-9ff0-4f800707bb48 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1546 microversion: 2.1 time: 0.253086 Aug 07 01:48:54.565771 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1234/2201] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:54 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1546 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:54.586784 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-646ec40e-6b61-4d1e-a233-d19a029bab2f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:54.587403 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-646ec40e-6b61-4d1e-a233-d19a029bab2f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:54.595456 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-646ec40e-6b61-4d1e-a233-d19a029bab2f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:54.596832 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-646ec40e-6b61-4d1e-a233-d19a029bab2f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:54.596832 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-646ec40e-6b61-4d1e-a233-d19a029bab2f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:54.597140 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-646ec40e-6b61-4d1e-a233-d19a029bab2f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:54.597581 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-646ec40e-6b61-4d1e-a233-d19a029bab2f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:54.598015 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-646ec40e-6b61-4d1e-a233-d19a029bab2f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:54.767878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-646ec40e-6b61-4d1e-a233-d19a029bab2f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:54.776660 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-646ec40e-6b61-4d1e-a233-d19a029bab2f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1546 microversion: 2.1 time: 0.193603 Aug 07 01:48:54.777153 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 968/2202] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:54 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1546 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:54.798050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-300e68f6-d305-4240-9248-8a5c5c0154ba tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:54.803304 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-300e68f6-d305-4240-9248-8a5c5c0154ba tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:54.804098 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-300e68f6-d305-4240-9248-8a5c5c0154ba tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Searching by: {'deleted': False, 'project_id': '673afe3c625d419e89fe80f3db4574fb'} {{(pid=117983) get_all /opt/stack/nova/nova/compute/api.py:2964}} Aug 07 01:48:54.811522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-300e68f6-d305-4240-9248-8a5c5c0154ba tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:54.812012 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-300e68f6-d305-4240-9248-8a5c5c0154ba tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:54.812485 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-300e68f6-d305-4240-9248-8a5c5c0154ba tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:54.813241 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-300e68f6-d305-4240-9248-8a5c5c0154ba tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:54.813701 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-300e68f6-d305-4240-9248-8a5c5c0154ba tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:54.814313 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-300e68f6-d305-4240-9248-8a5c5c0154ba tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:54.853967 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8439a234-bcb4-4297-9f02-487405c62c99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:54.855174 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8439a234-bcb4-4297-9f02-487405c62c99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:54.867450 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8439a234-bcb4-4297-9f02-487405c62c99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:54.868470 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8439a234-bcb4-4297-9f02-487405c62c99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:54.869072 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8439a234-bcb4-4297-9f02-487405c62c99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:54.870276 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8439a234-bcb4-4297-9f02-487405c62c99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:54.871034 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8439a234-bcb4-4297-9f02-487405c62c99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:54.872124 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8439a234-bcb4-4297-9f02-487405c62c99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:54.915493 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.multi_cell_list [None req-300e68f6-d305-4240-9248-8a5c5c0154ba tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=117983) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Aug 07 01:48:55.062761 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-300e68f6-d305-4240-9248-8a5c5c0154ba tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 342 microversion: 2.1 time: 0.269337 Aug 07 01:48:55.063844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1235/2203] 10.222.0.26 () {58 vars in 1125 bytes} [Sun Aug 7 01:48:54 2022] GET /compute/v2.1/servers => generated 342 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:48:55.082870 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6e5d7f87-00c0-4b12-92d3-1c8311e5d88d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:55.083941 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6e5d7f87-00c0-4b12-92d3-1c8311e5d88d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:55.092046 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6e5d7f87-00c0-4b12-92d3-1c8311e5d88d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:55.093293 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6e5d7f87-00c0-4b12-92d3-1c8311e5d88d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:55.094766 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6e5d7f87-00c0-4b12-92d3-1c8311e5d88d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:55.096022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6e5d7f87-00c0-4b12-92d3-1c8311e5d88d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:55.096892 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6e5d7f87-00c0-4b12-92d3-1c8311e5d88d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:55.097496 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6e5d7f87-00c0-4b12-92d3-1c8311e5d88d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:55.104976 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8439a234-bcb4-4297-9f02-487405c62c99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:55.107588 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=87 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:48:55.118490 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8439a234-bcb4-4297-9f02-487405c62c99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1261 microversion: 2.1 time: 0.272985 Aug 07 01:48:55.119486 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 969/2204] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:54 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1261 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:55.283815 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6e5d7f87-00c0-4b12-92d3-1c8311e5d88d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:55.297154 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6e5d7f87-00c0-4b12-92d3-1c8311e5d88d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1546 microversion: 2.1 time: 0.220998 Aug 07 01:48:55.297849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1236/2205] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:55 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1546 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:48:56.156519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-61dcc9e6-9a0c-4f3f-8b14-757b64f52bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:56.157425 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-61dcc9e6-9a0c-4f3f-8b14-757b64f52bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:56.160046 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=61,nova_api:INSERT=4 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:48:56.165952 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61dcc9e6-9a0c-4f3f-8b14-757b64f52bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:56.166487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61dcc9e6-9a0c-4f3f-8b14-757b64f52bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:56.166913 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61dcc9e6-9a0c-4f3f-8b14-757b64f52bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:56.167629 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61dcc9e6-9a0c-4f3f-8b14-757b64f52bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:56.168073 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61dcc9e6-9a0c-4f3f-8b14-757b64f52bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:56.168533 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-61dcc9e6-9a0c-4f3f-8b14-757b64f52bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:56.329871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-61dcc9e6-9a0c-4f3f-8b14-757b64f52bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:56.343942 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-61dcc9e6-9a0c-4f3f-8b14-757b64f52bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1261 microversion: 2.1 time: 0.191638 Aug 07 01:48:56.345271 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 970/2206] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:56 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1261 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:57.037951 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1d8327e1-75af-4b1d-8b34-64dc450dc951 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "58d33a82-ad11-4f6b-a622-e1732cd665ad"}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:48:57.041264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1d8327e1-75af-4b1d-8b34-64dc450dc951 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:57.043170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=6 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:48:57.049877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1d8327e1-75af-4b1d-8b34-64dc450dc951 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:57.050551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1d8327e1-75af-4b1d-8b34-64dc450dc951 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:57.051067 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1d8327e1-75af-4b1d-8b34-64dc450dc951 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:57.051996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1d8327e1-75af-4b1d-8b34-64dc450dc951 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:57.052589 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1d8327e1-75af-4b1d-8b34-64dc450dc951 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:57.053152 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1d8327e1-75af-4b1d-8b34-64dc450dc951 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:57.380314 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fc01797e-467a-4600-a427-18c9fa41ed18 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:57.381372 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fc01797e-467a-4600-a427-18c9fa41ed18 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:57.386828 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell0:SELECT=2 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:48:57.392346 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fc01797e-467a-4600-a427-18c9fa41ed18 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:57.393154 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fc01797e-467a-4600-a427-18c9fa41ed18 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:57.394062 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fc01797e-467a-4600-a427-18c9fa41ed18 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:57.395286 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fc01797e-467a-4600-a427-18c9fa41ed18 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:57.396067 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fc01797e-467a-4600-a427-18c9fa41ed18 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:57.396836 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fc01797e-467a-4600-a427-18c9fa41ed18 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:57.404213 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1d8327e1-75af-4b1d-8b34-64dc450dc951 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "POST /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.369768 Aug 07 01:48:57.405805 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1237/2207] 10.222.0.26 () {60 vars in 1323 bytes} [Sun Aug 7 01:48:57 2022] POST /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/os-volume_attachments => generated 194 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:48:57.671544 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fc01797e-467a-4600-a427-18c9fa41ed18 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:57.680363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fc01797e-467a-4600-a427-18c9fa41ed18 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1261 microversion: 2.1 time: 0.307805 Aug 07 01:48:57.680961 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 971/2208] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:57 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1261 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:48:58.705690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-eb36a48a-6096-4c5d-a39c-bd5e676aca80 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:58.706776 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-eb36a48a-6096-4c5d-a39c-bd5e676aca80 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:58.714824 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb36a48a-6096-4c5d-a39c-bd5e676aca80 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:58.715549 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb36a48a-6096-4c5d-a39c-bd5e676aca80 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:58.716092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb36a48a-6096-4c5d-a39c-bd5e676aca80 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:58.717030 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb36a48a-6096-4c5d-a39c-bd5e676aca80 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:58.719141 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb36a48a-6096-4c5d-a39c-bd5e676aca80 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:58.719702 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-eb36a48a-6096-4c5d-a39c-bd5e676aca80 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:58.933789 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-eb36a48a-6096-4c5d-a39c-bd5e676aca80 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:48:58.944666 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-eb36a48a-6096-4c5d-a39c-bd5e676aca80 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1261 microversion: 2.1 time: 0.244715 Aug 07 01:48:58.945521 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1238/2209] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:58 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1261 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:48:59.968554 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-261ae86b-4a30-4772-b10f-01c8a7297b93 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:48:59.969624 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-261ae86b-4a30-4772-b10f-01c8a7297b93 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:48:59.980500 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-261ae86b-4a30-4772-b10f-01c8a7297b93 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:59.981332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-261ae86b-4a30-4772-b10f-01c8a7297b93 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:59.982050 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-261ae86b-4a30-4772-b10f-01c8a7297b93 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:48:59.983819 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-261ae86b-4a30-4772-b10f-01c8a7297b93 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:48:59.984638 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-261ae86b-4a30-4772-b10f-01c8a7297b93 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:48:59.985313 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-261ae86b-4a30-4772-b10f-01c8a7297b93 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:00.176419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-261ae86b-4a30-4772-b10f-01c8a7297b93 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:00.186467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-261ae86b-4a30-4772-b10f-01c8a7297b93 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1261 microversion: 2.1 time: 0.222458 Aug 07 01:49:00.187673 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 972/2210] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:48:59 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1261 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:01.219690 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a4b621a3-e40a-4002-a6d2-c42ba34da7dc tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:01.220341 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a4b621a3-e40a-4002-a6d2-c42ba34da7dc tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:01.222922 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=61,nova_api:INSERT=4 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:49:01.228832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4b621a3-e40a-4002-a6d2-c42ba34da7dc tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:01.229340 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4b621a3-e40a-4002-a6d2-c42ba34da7dc tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:01.229810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4b621a3-e40a-4002-a6d2-c42ba34da7dc tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:01.230528 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4b621a3-e40a-4002-a6d2-c42ba34da7dc tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:01.231016 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4b621a3-e40a-4002-a6d2-c42ba34da7dc tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:01.231449 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a4b621a3-e40a-4002-a6d2-c42ba34da7dc tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:01.236044 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:49:01.392052 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a4b621a3-e40a-4002-a6d2-c42ba34da7dc tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:01.400483 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a4b621a3-e40a-4002-a6d2-c42ba34da7dc tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1261 microversion: 2.1 time: 0.184886 Aug 07 01:49:01.401301 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1239/2211] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:01 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1261 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:02.436156 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cd1b35da-b17d-4cc0-b8e4-aeac36a8bdfe tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:02.437032 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-cd1b35da-b17d-4cc0-b8e4-aeac36a8bdfe tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:02.447432 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd1b35da-b17d-4cc0-b8e4-aeac36a8bdfe tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:02.448332 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd1b35da-b17d-4cc0-b8e4-aeac36a8bdfe tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:02.449115 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd1b35da-b17d-4cc0-b8e4-aeac36a8bdfe tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:02.450395 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd1b35da-b17d-4cc0-b8e4-aeac36a8bdfe tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:02.451312 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd1b35da-b17d-4cc0-b8e4-aeac36a8bdfe tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:02.452836 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cd1b35da-b17d-4cc0-b8e4-aeac36a8bdfe tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:02.648420 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-cd1b35da-b17d-4cc0-b8e4-aeac36a8bdfe tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:02.657333 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-cd1b35da-b17d-4cc0-b8e4-aeac36a8bdfe tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1261 microversion: 2.1 time: 0.225913 Aug 07 01:49:02.657935 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 973/2212] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:02 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1261 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:03.682966 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-346afb9f-6d5d-4d85-a65b-e8f3a18d8ee1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:03.684087 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-346afb9f-6d5d-4d85-a65b-e8f3a18d8ee1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:03.696032 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-346afb9f-6d5d-4d85-a65b-e8f3a18d8ee1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:03.696956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-346afb9f-6d5d-4d85-a65b-e8f3a18d8ee1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:03.697738 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-346afb9f-6d5d-4d85-a65b-e8f3a18d8ee1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:03.699024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-346afb9f-6d5d-4d85-a65b-e8f3a18d8ee1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:03.699843 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-346afb9f-6d5d-4d85-a65b-e8f3a18d8ee1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:03.700664 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-346afb9f-6d5d-4d85-a65b-e8f3a18d8ee1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:03.913186 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-346afb9f-6d5d-4d85-a65b-e8f3a18d8ee1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:03.923059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-346afb9f-6d5d-4d85-a65b-e8f3a18d8ee1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1261 microversion: 2.1 time: 0.247412 Aug 07 01:49:03.923658 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1240/2213] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:03 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1261 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:04.188668 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-492c5e15-115d-47dc-900b-6780e4d75a89 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:04.189758 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-492c5e15-115d-47dc-900b-6780e4d75a89 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:04.201322 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-492c5e15-115d-47dc-900b-6780e4d75a89 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:04.202215 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-492c5e15-115d-47dc-900b-6780e4d75a89 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:04.202975 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-492c5e15-115d-47dc-900b-6780e4d75a89 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:04.204168 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-492c5e15-115d-47dc-900b-6780e4d75a89 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:04.205006 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-492c5e15-115d-47dc-900b-6780e4d75a89 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:04.205774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-492c5e15-115d-47dc-900b-6780e4d75a89 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:04.360899 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-492c5e15-115d-47dc-900b-6780e4d75a89 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:04.374675 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-492c5e15-115d-47dc-900b-6780e4d75a89 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1592 microversion: 2.1 time: 0.194155 Aug 07 01:49:04.375564 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 974/2214] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:04 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1592 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:04.739086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-623c8e35-8934-4f90-8d4f-b74655e9afc3 req-25440d34-9b8e-4f9c-88c9-0224566642a7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "768918d9-86a2-4c55-ba2a-57f5c1dcd234", "name": "network-vif-plugged", "status": "completed", "tag": "ebb91259-4e43-449e-b5bb-c646943874ee"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:49:04.752660 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-623c8e35-8934-4f90-8d4f-b74655e9afc3 req-25440d34-9b8e-4f9c-88c9-0224566642a7 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:04.753482 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-623c8e35-8934-4f90-8d4f-b74655e9afc3 req-25440d34-9b8e-4f9c-88c9-0224566642a7 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:04.753994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-623c8e35-8934-4f90-8d4f-b74655e9afc3 req-25440d34-9b8e-4f9c-88c9-0224566642a7 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:04.780499 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-623c8e35-8934-4f90-8d4f-b74655e9afc3 req-25440d34-9b8e-4f9c-88c9-0224566642a7 service nova] Creating event network-vif-plugged:ebb91259-4e43-449e-b5bb-c646943874ee for instance 768918d9-86a2-4c55-ba2a-57f5c1dcd234 on hv-cinder-maste Aug 07 01:49:04.793479 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-623c8e35-8934-4f90-8d4f-b74655e9afc3 req-25440d34-9b8e-4f9c-88c9-0224566642a7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056567 Aug 07 01:49:04.794634 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1241/2215] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:49:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:49:04.961715 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-3b2ece82-0a86-4e36-88e7-82bb6c57b378 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:04.962472 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-3b2ece82-0a86-4e36-88e7-82bb6c57b378 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:04.974347 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b2ece82-0a86-4e36-88e7-82bb6c57b378 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:04.975977 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b2ece82-0a86-4e36-88e7-82bb6c57b378 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:04.976873 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b2ece82-0a86-4e36-88e7-82bb6c57b378 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:04.978532 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b2ece82-0a86-4e36-88e7-82bb6c57b378 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:04.979859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b2ece82-0a86-4e36-88e7-82bb6c57b378 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:04.981145 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-3b2ece82-0a86-4e36-88e7-82bb6c57b378 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:05.382774 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-3b2ece82-0a86-4e36-88e7-82bb6c57b378 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:05.391725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-3b2ece82-0a86-4e36-88e7-82bb6c57b378 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1261 microversion: 2.1 time: 0.433990 Aug 07 01:49:05.392583 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 975/2216] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:04 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1261 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:06.428905 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell0:SELECT=4 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:49:06.439560 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-78d21e58-ef60-4770-8bad-f0f0eb668172 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:06.441226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-78d21e58-ef60-4770-8bad-f0f0eb668172 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:06.458056 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78d21e58-ef60-4770-8bad-f0f0eb668172 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:06.459617 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78d21e58-ef60-4770-8bad-f0f0eb668172 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:06.460967 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78d21e58-ef60-4770-8bad-f0f0eb668172 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:06.463296 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78d21e58-ef60-4770-8bad-f0f0eb668172 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:06.464677 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78d21e58-ef60-4770-8bad-f0f0eb668172 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:06.465742 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-78d21e58-ef60-4770-8bad-f0f0eb668172 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:06.722671 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-78d21e58-ef60-4770-8bad-f0f0eb668172 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:06.735255 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-78d21e58-ef60-4770-8bad-f0f0eb668172 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1261 microversion: 2.1 time: 0.308451 Aug 07 01:49:06.736196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1242/2217] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:06 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1261 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:07.141126 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-04f74722-a57b-4a24-8baa-60a420506444 req-dbc34b2d-c6c7-48fc-939d-b3cd811ba210 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "65635777-b502-4848-8327-c39ccd195e5d", "tag": "7fe8e7c8-c5ab-41e4-9fdb-393c265c2479"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:49:07.160329 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-04f74722-a57b-4a24-8baa-60a420506444 req-dbc34b2d-c6c7-48fc-939d-b3cd811ba210 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:07.161278 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-04f74722-a57b-4a24-8baa-60a420506444 req-dbc34b2d-c6c7-48fc-939d-b3cd811ba210 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:07.162345 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-04f74722-a57b-4a24-8baa-60a420506444 req-dbc34b2d-c6c7-48fc-939d-b3cd811ba210 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:07.195859 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-04f74722-a57b-4a24-8baa-60a420506444 req-dbc34b2d-c6c7-48fc-939d-b3cd811ba210 service nova] Creating event network-changed:7fe8e7c8-c5ab-41e4-9fdb-393c265c2479 for instance 65635777-b502-4848-8327-c39ccd195e5d on hv-cinder-maste Aug 07 01:49:07.205544 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-04f74722-a57b-4a24-8baa-60a420506444 req-dbc34b2d-c6c7-48fc-939d-b3cd811ba210 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070761 Aug 07 01:49:07.206666 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 976/2218] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:49:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:49:07.771902 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-64109dc5-417f-408f-9b03-63f95ae1aa60 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:07.775170 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-64109dc5-417f-408f-9b03-63f95ae1aa60 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:07.796420 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-64109dc5-417f-408f-9b03-63f95ae1aa60 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:07.797325 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-64109dc5-417f-408f-9b03-63f95ae1aa60 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:07.798242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-64109dc5-417f-408f-9b03-63f95ae1aa60 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:07.802277 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-64109dc5-417f-408f-9b03-63f95ae1aa60 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:07.803673 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-64109dc5-417f-408f-9b03-63f95ae1aa60 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:07.804763 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-64109dc5-417f-408f-9b03-63f95ae1aa60 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:08.183002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-64109dc5-417f-408f-9b03-63f95ae1aa60 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:08.194923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-64109dc5-417f-408f-9b03-63f95ae1aa60 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1445 microversion: 2.1 time: 0.438051 Aug 07 01:49:08.196222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1243/2219] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:07 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1445 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:08.236840 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9690ae7b-ff67-40c1-a532-fbe17428e50b tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:08.239279 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9690ae7b-ff67-40c1-a532-fbe17428e50b tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:08.251938 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9690ae7b-ff67-40c1-a532-fbe17428e50b tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:08.253236 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9690ae7b-ff67-40c1-a532-fbe17428e50b tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:08.254224 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9690ae7b-ff67-40c1-a532-fbe17428e50b tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:08.255520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9690ae7b-ff67-40c1-a532-fbe17428e50b tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:08.256698 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9690ae7b-ff67-40c1-a532-fbe17428e50b tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:08.257929 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9690ae7b-ff67-40c1-a532-fbe17428e50b tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:08.543524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9690ae7b-ff67-40c1-a532-fbe17428e50b tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:08.560272 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9690ae7b-ff67-40c1-a532-fbe17428e50b tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1445 microversion: 2.1 time: 0.331858 Aug 07 01:49:08.562211 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 977/2220] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:08 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1445 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:09.358727 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-97fc67f5-b32d-4c2c-9e33-45b1d44c29ab tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-272623057"}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:49:09.359248 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-97fc67f5-b32d-4c2c-9e33-45b1d44c29ab tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:09.368284 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97fc67f5-b32d-4c2c-9e33-45b1d44c29ab tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:09.368799 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97fc67f5-b32d-4c2c-9e33-45b1d44c29ab tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:09.369250 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97fc67f5-b32d-4c2c-9e33-45b1d44c29ab tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:09.370059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97fc67f5-b32d-4c2c-9e33-45b1d44c29ab tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:09.370654 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97fc67f5-b32d-4c2c-9e33-45b1d44c29ab tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:09.371005 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-97fc67f5-b32d-4c2c-9e33-45b1d44c29ab tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:09.654911 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.network.security_group_api [None req-97fc67f5-b32d-4c2c-9e33-45b1d44c29ab tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Adding security group 303250ab-e207-4c74-8766-c52e88606dc9 to port 7fe8e7c8-c5ab-41e4-9fdb-393c265c2479 Aug 07 01:49:10.457932 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-97fc67f5-b32d-4c2c-9e33-45b1d44c29ab tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "POST /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/action" status: 202 len: 0 microversion: 2.1 time: 1.104310 Aug 07 01:49:10.459689 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1244/2221] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:49:09 2022] POST /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/action => generated 0 bytes in 1107 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 07 01:49:10.486877 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8ab84f69-5697-4286-9f31-f2ac2b71672a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:10.489179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8ab84f69-5697-4286-9f31-f2ac2b71672a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:10.507273 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ab84f69-5697-4286-9f31-f2ac2b71672a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:10.508494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ab84f69-5697-4286-9f31-f2ac2b71672a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:10.509495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ab84f69-5697-4286-9f31-f2ac2b71672a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:10.512119 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ab84f69-5697-4286-9f31-f2ac2b71672a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:10.513212 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ab84f69-5697-4286-9f31-f2ac2b71672a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:10.514856 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ab84f69-5697-4286-9f31-f2ac2b71672a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:10.800108 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-7d80096a-e86c-4c00-abfa-a0e43c43d06e req-a0cc1c93-0805-47f4-94f2-3304d5218c60 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "65635777-b502-4848-8327-c39ccd195e5d", "tag": "7fe8e7c8-c5ab-41e4-9fdb-393c265c2479"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:49:10.839306 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-7d80096a-e86c-4c00-abfa-a0e43c43d06e req-a0cc1c93-0805-47f4-94f2-3304d5218c60 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:10.841215 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-7d80096a-e86c-4c00-abfa-a0e43c43d06e req-a0cc1c93-0805-47f4-94f2-3304d5218c60 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:10.842876 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-7d80096a-e86c-4c00-abfa-a0e43c43d06e req-a0cc1c93-0805-47f4-94f2-3304d5218c60 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:10.872179 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-7d80096a-e86c-4c00-abfa-a0e43c43d06e req-a0cc1c93-0805-47f4-94f2-3304d5218c60 service nova] Creating event network-changed:7fe8e7c8-c5ab-41e4-9fdb-393c265c2479 for instance 65635777-b502-4848-8327-c39ccd195e5d on hv-cinder-maste Aug 07 01:49:10.884036 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-7d80096a-e86c-4c00-abfa-a0e43c43d06e req-a0cc1c93-0805-47f4-94f2-3304d5218c60 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.405658 Aug 07 01:49:10.887673 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1245/2222] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:49:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:49:11.559780 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8ab84f69-5697-4286-9f31-f2ac2b71672a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:11.578095 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8ab84f69-5697-4286-9f31-f2ac2b71672a tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1757 microversion: 2.1 time: 1.109159 Aug 07 01:49:11.579543 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 978/2223] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:10 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1757 bytes in 1113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:12.907561 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-c67dfa03-3466-4d49-a316-351bdb1e90d1 req-3b248363-c616-44f2-a0cc-cf2139d9043d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "768918d9-86a2-4c55-ba2a-57f5c1dcd234", "tag": "ebb91259-4e43-449e-b5bb-c646943874ee"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:49:12.923562 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-c67dfa03-3466-4d49-a316-351bdb1e90d1 req-3b248363-c616-44f2-a0cc-cf2139d9043d service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:12.924713 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-c67dfa03-3466-4d49-a316-351bdb1e90d1 req-3b248363-c616-44f2-a0cc-cf2139d9043d service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:12.925770 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-c67dfa03-3466-4d49-a316-351bdb1e90d1 req-3b248363-c616-44f2-a0cc-cf2139d9043d service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:12.951138 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-c67dfa03-3466-4d49-a316-351bdb1e90d1 req-3b248363-c616-44f2-a0cc-cf2139d9043d service nova] Creating event network-changed:ebb91259-4e43-449e-b5bb-c646943874ee for instance 768918d9-86a2-4c55-ba2a-57f5c1dcd234 on hv-cinder-maste Aug 07 01:49:12.960832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-c67dfa03-3466-4d49-a316-351bdb1e90d1 req-3b248363-c616-44f2-a0cc-cf2139d9043d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058948 Aug 07 01:49:12.961810 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1246/2224] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:49:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:49:34.361024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-63683a35-7e81-4b37-9042-e692cdcd38f8 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:49:34.364014 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-63683a35-7e81-4b37-9042-e692cdcd38f8 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:34.367124 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=17 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:49:34.368347 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=51 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:49:34.377389 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-63683a35-7e81-4b37-9042-e692cdcd38f8 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:34.378148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-63683a35-7e81-4b37-9042-e692cdcd38f8 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:34.378893 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-63683a35-7e81-4b37-9042-e692cdcd38f8 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:34.380070 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-63683a35-7e81-4b37-9042-e692cdcd38f8 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:34.380868 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-63683a35-7e81-4b37-9042-e692cdcd38f8 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:34.381594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-63683a35-7e81-4b37-9042-e692cdcd38f8 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:34.488132 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-63683a35-7e81-4b37-9042-e692cdcd38f8 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "POST /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/action" status: 202 len: 0 microversion: 2.1 time: 0.132918 Aug 07 01:49:34.488993 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 979/2225] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:49:34 2022] POST /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/action => generated 0 bytes in 135 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 07 01:49:34.506530 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-1505f38e-eb83-4f42-9fbc-32a74b3c6b5f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:34.507743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-1505f38e-eb83-4f42-9fbc-32a74b3c6b5f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:34.513850 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=15 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:49:34.515075 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=33 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:49:34.523134 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1505f38e-eb83-4f42-9fbc-32a74b3c6b5f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:34.523694 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1505f38e-eb83-4f42-9fbc-32a74b3c6b5f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:34.524076 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1505f38e-eb83-4f42-9fbc-32a74b3c6b5f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:34.524839 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1505f38e-eb83-4f42-9fbc-32a74b3c6b5f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:34.525297 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1505f38e-eb83-4f42-9fbc-32a74b3c6b5f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:34.525724 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-1505f38e-eb83-4f42-9fbc-32a74b3c6b5f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:34.758041 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-1505f38e-eb83-4f42-9fbc-32a74b3c6b5f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:34.766969 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-1505f38e-eb83-4f42-9fbc-32a74b3c6b5f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1749 microversion: 2.1 time: 0.270821 Aug 07 01:49:34.767671 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1247/2226] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:34 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1749 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:35.802186 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-01aa464e-a127-4880-bed4-d6f26323f354 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:35.802890 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-01aa464e-a127-4880-bed4-d6f26323f354 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:35.811782 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-01aa464e-a127-4880-bed4-d6f26323f354 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:35.812288 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-01aa464e-a127-4880-bed4-d6f26323f354 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:35.812750 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-01aa464e-a127-4880-bed4-d6f26323f354 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:35.813526 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-01aa464e-a127-4880-bed4-d6f26323f354 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:35.813977 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-01aa464e-a127-4880-bed4-d6f26323f354 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:35.814400 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-01aa464e-a127-4880-bed4-d6f26323f354 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:35.986264 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-01aa464e-a127-4880-bed4-d6f26323f354 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:36.001707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-01aa464e-a127-4880-bed4-d6f26323f354 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1754 microversion: 2.1 time: 0.203355 Aug 07 01:49:36.002785 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 980/2227] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:35 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1754 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:49:37.028919 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5d6111f9-3d7f-4633-8373-618efcf67246 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:37.029570 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5d6111f9-3d7f-4633-8373-618efcf67246 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:37.036149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d6111f9-3d7f-4633-8373-618efcf67246 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:37.036696 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d6111f9-3d7f-4633-8373-618efcf67246 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:37.037174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d6111f9-3d7f-4633-8373-618efcf67246 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:37.037963 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d6111f9-3d7f-4633-8373-618efcf67246 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:37.038426 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d6111f9-3d7f-4633-8373-618efcf67246 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:37.038851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5d6111f9-3d7f-4633-8373-618efcf67246 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:37.199716 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5d6111f9-3d7f-4633-8373-618efcf67246 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:37.214869 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5d6111f9-3d7f-4633-8373-618efcf67246 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1754 microversion: 2.1 time: 0.189404 Aug 07 01:49:37.215933 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1248/2228] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:37 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1754 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:38.238451 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cfbc4ad2-d876-43f7-a982-9402a8a4d5bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:38.239202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-cfbc4ad2-d876-43f7-a982-9402a8a4d5bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:38.247276 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cfbc4ad2-d876-43f7-a982-9402a8a4d5bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:38.248489 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cfbc4ad2-d876-43f7-a982-9402a8a4d5bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:38.248996 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cfbc4ad2-d876-43f7-a982-9402a8a4d5bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:38.249781 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cfbc4ad2-d876-43f7-a982-9402a8a4d5bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:38.250276 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cfbc4ad2-d876-43f7-a982-9402a8a4d5bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:38.250748 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cfbc4ad2-d876-43f7-a982-9402a8a4d5bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:38.416103 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-cfbc4ad2-d876-43f7-a982-9402a8a4d5bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:38.425494 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-cfbc4ad2-d876-43f7-a982-9402a8a4d5bb tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1754 microversion: 2.1 time: 0.190981 Aug 07 01:49:38.426088 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 981/2229] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:38 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1754 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:39.459407 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-8e544299-db76-45c4-b8fc-152c48adb03e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:39.460157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-8e544299-db76-45c4-b8fc-152c48adb03e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:39.468604 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e544299-db76-45c4-b8fc-152c48adb03e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:39.469231 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e544299-db76-45c4-b8fc-152c48adb03e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:39.469765 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e544299-db76-45c4-b8fc-152c48adb03e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:39.470614 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e544299-db76-45c4-b8fc-152c48adb03e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:39.471165 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e544299-db76-45c4-b8fc-152c48adb03e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:39.471686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-8e544299-db76-45c4-b8fc-152c48adb03e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:39.637477 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-8e544299-db76-45c4-b8fc-152c48adb03e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:39.646947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-8e544299-db76-45c4-b8fc-152c48adb03e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1754 microversion: 2.1 time: 0.192386 Aug 07 01:49:39.647552 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1249/2230] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:39 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1754 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:40.670202 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f706829e-d88f-4d8d-88e7-3fa7904b0e06 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:40.670793 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f706829e-d88f-4d8d-88e7-3fa7904b0e06 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:40.677605 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f706829e-d88f-4d8d-88e7-3fa7904b0e06 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:40.678237 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f706829e-d88f-4d8d-88e7-3fa7904b0e06 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:40.678672 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f706829e-d88f-4d8d-88e7-3fa7904b0e06 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:40.679379 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f706829e-d88f-4d8d-88e7-3fa7904b0e06 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:40.679829 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f706829e-d88f-4d8d-88e7-3fa7904b0e06 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:40.680315 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f706829e-d88f-4d8d-88e7-3fa7904b0e06 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:40.865087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f706829e-d88f-4d8d-88e7-3fa7904b0e06 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:40.873775 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f706829e-d88f-4d8d-88e7-3fa7904b0e06 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1754 microversion: 2.1 time: 0.207873 Aug 07 01:49:40.874475 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 982/2231] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:40 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1754 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:49:41.897858 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-da0d02d1-f192-443e-a8d5-9389e7b8516e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:41.898463 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-da0d02d1-f192-443e-a8d5-9389e7b8516e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:41.905718 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-da0d02d1-f192-443e-a8d5-9389e7b8516e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:41.906226 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-da0d02d1-f192-443e-a8d5-9389e7b8516e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:41.906658 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-da0d02d1-f192-443e-a8d5-9389e7b8516e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:41.907355 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-da0d02d1-f192-443e-a8d5-9389e7b8516e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:41.907831 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-da0d02d1-f192-443e-a8d5-9389e7b8516e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:41.908272 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-da0d02d1-f192-443e-a8d5-9389e7b8516e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:42.078409 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-da0d02d1-f192-443e-a8d5-9389e7b8516e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:42.087554 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-da0d02d1-f192-443e-a8d5-9389e7b8516e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1754 microversion: 2.1 time: 0.193446 Aug 07 01:49:42.088447 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1250/2232] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:41 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1754 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:43.111106 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-b720f223-c53f-4df0-85b6-4afc811e0800 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:43.111949 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-b720f223-c53f-4df0-85b6-4afc811e0800 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:43.118725 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b720f223-c53f-4df0-85b6-4afc811e0800 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:43.119231 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b720f223-c53f-4df0-85b6-4afc811e0800 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:43.119665 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b720f223-c53f-4df0-85b6-4afc811e0800 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:43.120419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b720f223-c53f-4df0-85b6-4afc811e0800 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:43.120863 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b720f223-c53f-4df0-85b6-4afc811e0800 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:43.121282 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-b720f223-c53f-4df0-85b6-4afc811e0800 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:43.301367 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-b720f223-c53f-4df0-85b6-4afc811e0800 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:43.316932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-b720f223-c53f-4df0-85b6-4afc811e0800 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1754 microversion: 2.1 time: 0.209527 Aug 07 01:49:43.317917 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 983/2233] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:43 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1754 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:44.355877 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-54b16ad5-548d-4945-83e0-e3aabfa49d31 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:44.357006 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-54b16ad5-548d-4945-83e0-e3aabfa49d31 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:44.370623 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54b16ad5-548d-4945-83e0-e3aabfa49d31 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:44.371568 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54b16ad5-548d-4945-83e0-e3aabfa49d31 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:44.372430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54b16ad5-548d-4945-83e0-e3aabfa49d31 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:44.374298 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54b16ad5-548d-4945-83e0-e3aabfa49d31 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:44.375130 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54b16ad5-548d-4945-83e0-e3aabfa49d31 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:44.375886 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-54b16ad5-548d-4945-83e0-e3aabfa49d31 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:44.551817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-54b16ad5-548d-4945-83e0-e3aabfa49d31 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:44.567549 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-54b16ad5-548d-4945-83e0-e3aabfa49d31 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1754 microversion: 2.1 time: 0.219609 Aug 07 01:49:44.568264 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1251/2234] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:44 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1754 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:45.594871 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-fbd58df6-396f-4fd8-83e9-33a2be501820 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:45.595903 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-fbd58df6-396f-4fd8-83e9-33a2be501820 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:45.610894 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbd58df6-396f-4fd8-83e9-33a2be501820 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:45.612134 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbd58df6-396f-4fd8-83e9-33a2be501820 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:45.613249 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbd58df6-396f-4fd8-83e9-33a2be501820 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:45.614549 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbd58df6-396f-4fd8-83e9-33a2be501820 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:45.615383 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbd58df6-396f-4fd8-83e9-33a2be501820 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:45.616551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-fbd58df6-396f-4fd8-83e9-33a2be501820 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:45.783016 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-fbd58df6-396f-4fd8-83e9-33a2be501820 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:45.792016 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-fbd58df6-396f-4fd8-83e9-33a2be501820 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1754 microversion: 2.1 time: 0.205359 Aug 07 01:49:45.792697 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 984/2235] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:45 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1754 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:49:46.814743 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9371d6d6-0052-44f9-8db0-7fa32aba2365 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:49:46.815398 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9371d6d6-0052-44f9-8db0-7fa32aba2365 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:49:46.822193 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9371d6d6-0052-44f9-8db0-7fa32aba2365 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:46.822685 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9371d6d6-0052-44f9-8db0-7fa32aba2365 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:46.823125 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9371d6d6-0052-44f9-8db0-7fa32aba2365 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:46.823847 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9371d6d6-0052-44f9-8db0-7fa32aba2365 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:49:46.824332 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9371d6d6-0052-44f9-8db0-7fa32aba2365 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:49:46.824760 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9371d6d6-0052-44f9-8db0-7fa32aba2365 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:49:46.991625 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9371d6d6-0052-44f9-8db0-7fa32aba2365 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:49:47.001844 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9371d6d6-0052-44f9-8db0-7fa32aba2365 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1757 microversion: 2.1 time: 0.191449 Aug 07 01:49:47.002510 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1252/2236] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:49:46 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1757 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:50:22.295262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-07ecb2c3-c2a1-416a-a906-158d0011c53f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-272623057"}} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:50:22.295934 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-07ecb2c3-c2a1-416a-a906-158d0011c53f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:50:22.298262 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=12 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:50:22.299017 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:50:22.307130 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07ecb2c3-c2a1-416a-a906-158d0011c53f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:22.307628 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07ecb2c3-c2a1-416a-a906-158d0011c53f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:22.308063 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07ecb2c3-c2a1-416a-a906-158d0011c53f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:22.308826 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07ecb2c3-c2a1-416a-a906-158d0011c53f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:22.309272 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07ecb2c3-c2a1-416a-a906-158d0011c53f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:22.310701 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-07ecb2c3-c2a1-416a-a906-158d0011c53f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:22.463385 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.network.security_group_api [None req-07ecb2c3-c2a1-416a-a906-158d0011c53f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Removing security group 303250ab-e207-4c74-8766-c52e88606dc9 from port 7fe8e7c8-c5ab-41e4-9fdb-393c265c2479 Aug 07 01:50:22.873495 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-07ecb2c3-c2a1-416a-a906-158d0011c53f tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "POST /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/action" status: 202 len: 0 microversion: 2.1 time: 0.582205 Aug 07 01:50:22.874517 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 985/2237] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:50:22 2022] POST /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/action => generated 0 bytes in 584 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 07 01:50:22.881350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-b3ef5a35-5ca5-4d2d-8a29-7ab94de4642f req-7d4ccbe0-107e-440f-8a44-208f8729a445 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "65635777-b502-4848-8327-c39ccd195e5d", "tag": "7fe8e7c8-c5ab-41e4-9fdb-393c265c2479"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:50:22.888002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=12 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:50:22.888678 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=36 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:50:22.894001 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-b3ef5a35-5ca5-4d2d-8a29-7ab94de4642f req-7d4ccbe0-107e-440f-8a44-208f8729a445 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:22.894859 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-b3ef5a35-5ca5-4d2d-8a29-7ab94de4642f req-7d4ccbe0-107e-440f-8a44-208f8729a445 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:22.895304 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-b3ef5a35-5ca5-4d2d-8a29-7ab94de4642f req-7d4ccbe0-107e-440f-8a44-208f8729a445 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:22.911197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-b3ef5a35-5ca5-4d2d-8a29-7ab94de4642f req-7d4ccbe0-107e-440f-8a44-208f8729a445 service nova] Creating event network-changed:7fe8e7c8-c5ab-41e4-9fdb-393c265c2479 for instance 65635777-b502-4848-8327-c39ccd195e5d on hv-cinder-maste Aug 07 01:50:22.917367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-b3ef5a35-5ca5-4d2d-8a29-7ab94de4642f req-7d4ccbe0-107e-440f-8a44-208f8729a445 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039999 Aug 07 01:50:22.918430 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1253/2238] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:50:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:50:24.745849 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-02d9303d-1e6c-485f-ac09-365c013d1786 req-dc4c9e6c-0b22-4d43-9b49-c446292ee4a9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "65635777-b502-4848-8327-c39ccd195e5d", "tag": "7fe8e7c8-c5ab-41e4-9fdb-393c265c2479"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:50:24.756945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-19e9f678-d91e-449a-b60d-c0e3b97c2536 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:50:24.757941 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-19e9f678-d91e-449a-b60d-c0e3b97c2536 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:50:24.761188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-02d9303d-1e6c-485f-ac09-365c013d1786 req-dc4c9e6c-0b22-4d43-9b49-c446292ee4a9 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:24.761823 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-02d9303d-1e6c-485f-ac09-365c013d1786 req-dc4c9e6c-0b22-4d43-9b49-c446292ee4a9 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:24.762482 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-02d9303d-1e6c-485f-ac09-365c013d1786 req-dc4c9e6c-0b22-4d43-9b49-c446292ee4a9 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:24.767980 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19e9f678-d91e-449a-b60d-c0e3b97c2536 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:24.769003 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19e9f678-d91e-449a-b60d-c0e3b97c2536 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:24.769786 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19e9f678-d91e-449a-b60d-c0e3b97c2536 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:24.770956 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19e9f678-d91e-449a-b60d-c0e3b97c2536 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:24.771755 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19e9f678-d91e-449a-b60d-c0e3b97c2536 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:24.773002 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-19e9f678-d91e-449a-b60d-c0e3b97c2536 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:24.794422 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-02d9303d-1e6c-485f-ac09-365c013d1786 req-dc4c9e6c-0b22-4d43-9b49-c446292ee4a9 service nova] Creating event network-changed:7fe8e7c8-c5ab-41e4-9fdb-393c265c2479 for instance 65635777-b502-4848-8327-c39ccd195e5d on hv-cinder-maste Aug 07 01:50:24.800682 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-02d9303d-1e6c-485f-ac09-365c013d1786 req-dc4c9e6c-0b22-4d43-9b49-c446292ee4a9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058726 Aug 07 01:50:24.801686 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1254/2239] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:50:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:50:24.890047 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.objects.instance [None req-19e9f678-d91e-449a-b60d-c0e3b97c2536 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lazy-loading 'services' on Instance uuid 65635777-b502-4848-8327-c39ccd195e5d {{(pid=117984) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:50:24.988562 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-19e9f678-d91e-449a-b60d-c0e3b97c2536 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "DELETE /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/os-volume_attachments/58d33a82-ad11-4f6b-a622-e1732cd665ad" status: 202 len: 0 microversion: 2.1 time: 0.251946 Aug 07 01:50:24.992936 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 986/2240] 10.222.0.26 () {58 vars in 1416 bytes} [Sun Aug 7 01:50:24 2022] DELETE /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/os-volume_attachments/58d33a82-ad11-4f6b-a622-e1732cd665ad => generated 0 bytes in 258 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:50:25.005329 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-ab06d5b9-8ab8-4d84-beeb-caf8490c302c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:50:25.011962 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-ab06d5b9-8ab8-4d84-beeb-caf8490c302c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:50:25.048801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab06d5b9-8ab8-4d84-beeb-caf8490c302c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:25.050971 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab06d5b9-8ab8-4d84-beeb-caf8490c302c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:25.051988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab06d5b9-8ab8-4d84-beeb-caf8490c302c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:25.053512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab06d5b9-8ab8-4d84-beeb-caf8490c302c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:25.054611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab06d5b9-8ab8-4d84-beeb-caf8490c302c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:25.055734 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-ab06d5b9-8ab8-4d84-beeb-caf8490c302c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:25.113155 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-ab06d5b9-8ab8-4d84-beeb-caf8490c302c tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.117795 Aug 07 01:50:25.114157 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1255/2241] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:50:24 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/os-volume_attachments => generated 197 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:50:26.129953 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-89b60684-96e8-451c-ad39-e21829ffa12e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:50:26.132765 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-89b60684-96e8-451c-ad39-e21829ffa12e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:50:26.140389 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89b60684-96e8-451c-ad39-e21829ffa12e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:26.140878 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89b60684-96e8-451c-ad39-e21829ffa12e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:26.141327 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89b60684-96e8-451c-ad39-e21829ffa12e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:26.142054 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89b60684-96e8-451c-ad39-e21829ffa12e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:26.142491 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89b60684-96e8-451c-ad39-e21829ffa12e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:26.142925 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-89b60684-96e8-451c-ad39-e21829ffa12e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:26.175433 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-89b60684-96e8-451c-ad39-e21829ffa12e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.049922 Aug 07 01:50:26.176142 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 987/2242] 10.222.0.26 () {58 vars in 1302 bytes} [Sun Aug 7 01:50:26 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 07 01:50:26.186851 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0de33e74-3e64-453f-9e44-9fbe54ae52be tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:50:26.187994 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-0de33e74-3e64-453f-9e44-9fbe54ae52be tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:50:26.195071 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0de33e74-3e64-453f-9e44-9fbe54ae52be tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:26.195576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0de33e74-3e64-453f-9e44-9fbe54ae52be tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:26.196016 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0de33e74-3e64-453f-9e44-9fbe54ae52be tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:26.196840 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0de33e74-3e64-453f-9e44-9fbe54ae52be tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:26.197251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0de33e74-3e64-453f-9e44-9fbe54ae52be tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:26.197646 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-0de33e74-3e64-453f-9e44-9fbe54ae52be tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:26.289611 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-0de33e74-3e64-453f-9e44-9fbe54ae52be tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] HTTP exception thrown: Instance 65635777-b502-4848-8327-c39ccd195e5d is not attached to volume 58d33a82-ad11-4f6b-a622-e1732cd665ad Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0de33e74-3e64-453f-9e44-9fbe54ae52be tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Request method failure captured: Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: DELETE /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/os-volume_attachments/58d33a82-ad11-4f6b-a622-e1732cd665ad HTTP/1.1 Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xnPQSQsbHsL2SrVh9srh6F1pP5uHu5BFqaR2gUCkjKkPA8aUQ0lvE0puU1GDrZedNiBYbfTh0IAvzL8zy3w14tC4TpETWq7hGnm-b-hyulT7b5SABfBOapJ5fZEcYgC-O8twlz_rhbX3pUkxAn-4TWafkOqE4Ai6PfTkYdYqMU89Mc Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 673afe3c625d419e89fe80f3db4574fb Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-TestMinimumBasicScenario-301244876 Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:50:26.292026 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:50:26.293545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/673afe3c625d419e89fe80f3db4574fb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/673afe3c625d419e89fe80f3db4574fb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_673afe3c625d419e89fe80f3db4574fb"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/673afe3c625d419e89fe80f3db4574fb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:50:26.293545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-TestMinimumBasicScenario-301244876 Aug 07 01:50:26.293545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 673afe3c625d419e89fe80f3db4574fb Aug 07 01:50:26.293545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-TestMinimumBasicScenario-301244876 Aug 07 01:50:26.293545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-TestMinimumBasicScenario-301244876-project Aug 07 01:50:26.293545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:50:26.293545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:50:26.293545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: a11fb6edaae94f8ba2214e5ec0c65972 Aug 07 01:50:26.293545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-TestMinimumBasicScenario-301244876-project Aug 07 01:50:26.293545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:50:26.293545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 65635777-b502-4848-8327-c39ccd195e5d is not attached to volume 58d33a82-ad11-4f6b-a622-e1732cd665ad Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'server_id': '65635777-b502-4848-8327-c39ccd195e5d', 'id': '58d33a82-ad11-4f6b-a622-e1732cd665ad'} Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 281, in get_by_volume_and_instance Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 58d33a82-ad11-4f6b-a622-e1732cd665ad. Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:50:26.295196 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 65635777-b502-4848-8327-c39ccd195e5d is not attached to volume 58d33a82-ad11-4f6b-a622-e1732cd665ad Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 65635777-b502-4848-8327-c39ccd195e5d is not attached to volume 58d33a82-ad11-4f6b-a622-e1732cd665ad Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:50:26.297147 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-0de33e74-3e64-453f-9e44-9fbe54ae52be tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Returning 404 to user: Instance 65635777-b502-4848-8327-c39ccd195e5d is not attached to volume 58d33a82-ad11-4f6b-a622-e1732cd665ad {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:50:26.298082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-0de33e74-3e64-453f-9e44-9fbe54ae52be tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "DELETE /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/os-volume_attachments/58d33a82-ad11-4f6b-a622-e1732cd665ad" status: 404 len: 154 microversion: 2.1 time: 0.110846 Aug 07 01:50:26.298082 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1256/2243] 10.222.0.26 () {58 vars in 1416 bytes} [Sun Aug 7 01:50:26 2022] DELETE /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d/os-volume_attachments/58d33a82-ad11-4f6b-a622-e1732cd665ad => generated 154 bytes in 114 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 07 01:50:27.461623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-a5516318-6ad4-4031-8841-9cfccd94420d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:50:27.462289 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a5516318-6ad4-4031-8841-9cfccd94420d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:50:27.469181 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5516318-6ad4-4031-8841-9cfccd94420d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:27.469672 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5516318-6ad4-4031-8841-9cfccd94420d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:27.470108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5516318-6ad4-4031-8841-9cfccd94420d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:27.470818 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5516318-6ad4-4031-8841-9cfccd94420d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:27.471271 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5516318-6ad4-4031-8841-9cfccd94420d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:27.471698 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-a5516318-6ad4-4031-8841-9cfccd94420d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:27.499636 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-a5516318-6ad4-4031-8841-9cfccd94420d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Going to try to terminate instance {{(pid=117984) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:50:27.592510 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-a5516318-6ad4-4031-8841-9cfccd94420d tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "DELETE /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 204 len: 0 microversion: 2.1 time: 0.133802 Aug 07 01:50:27.594044 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 988/2244] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:50:27 2022] DELETE /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 07 01:50:27.606345 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-42eee0d8-b92f-4c34-b21d-0c82ae0e2a98 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:50:27.607896 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-42eee0d8-b92f-4c34-b21d-0c82ae0e2a98 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:50:27.618544 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-42eee0d8-b92f-4c34-b21d-0c82ae0e2a98 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:27.619387 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-42eee0d8-b92f-4c34-b21d-0c82ae0e2a98 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:27.620161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-42eee0d8-b92f-4c34-b21d-0c82ae0e2a98 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:27.621515 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-42eee0d8-b92f-4c34-b21d-0c82ae0e2a98 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:27.622956 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-42eee0d8-b92f-4c34-b21d-0c82ae0e2a98 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:27.623710 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-42eee0d8-b92f-4c34-b21d-0c82ae0e2a98 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:27.805582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-42eee0d8-b92f-4c34-b21d-0c82ae0e2a98 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:50:27.813786 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-42eee0d8-b92f-4c34-b21d-0c82ae0e2a98 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1552 microversion: 2.1 time: 0.214786 Aug 07 01:50:27.814583 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1257/2245] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:50:27 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1552 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:50:28.835844 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-e4277af6-ca6f-4568-9498-8cba98a7ab04 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:50:28.836591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-e4277af6-ca6f-4568-9498-8cba98a7ab04 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:50:28.844007 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4277af6-ca6f-4568-9498-8cba98a7ab04 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:28.844520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4277af6-ca6f-4568-9498-8cba98a7ab04 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:28.844972 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4277af6-ca6f-4568-9498-8cba98a7ab04 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:28.845696 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4277af6-ca6f-4568-9498-8cba98a7ab04 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:28.846157 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4277af6-ca6f-4568-9498-8cba98a7ab04 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:28.846579 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-e4277af6-ca6f-4568-9498-8cba98a7ab04 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:28.993133 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-e4277af6-ca6f-4568-9498-8cba98a7ab04 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:50:29.001379 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-e4277af6-ca6f-4568-9498-8cba98a7ab04 tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1552 microversion: 2.1 time: 0.169821 Aug 07 01:50:29.001974 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 989/2246] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:50:28 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1552 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:50:30.028144 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-e613d43f-9024-4691-8aa8-76555cf6e51e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:50:30.031440 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-e613d43f-9024-4691-8aa8-76555cf6e51e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:50:30.049947 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e613d43f-9024-4691-8aa8-76555cf6e51e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:30.050882 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e613d43f-9024-4691-8aa8-76555cf6e51e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:30.051704 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e613d43f-9024-4691-8aa8-76555cf6e51e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:30.052964 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e613d43f-9024-4691-8aa8-76555cf6e51e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:30.054027 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e613d43f-9024-4691-8aa8-76555cf6e51e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:30.055037 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-e613d43f-9024-4691-8aa8-76555cf6e51e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:30.203513 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-c7dce2b3-1e67-4014-be36-64b873445fc1 req-41bc5046-6941-4882-80fb-3a0e6f6a792f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "65635777-b502-4848-8327-c39ccd195e5d", "name": "network-vif-deleted", "tag": "7fe8e7c8-c5ab-41e4-9fdb-393c265c2479"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:50:30.204859 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-e613d43f-9024-4691-8aa8-76555cf6e51e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a11fb6edaae94f8ba2214e5ec0c65972', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '673afe3c625d419e89fe80f3db4574fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:50:30.220361 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c7dce2b3-1e67-4014-be36-64b873445fc1 req-41bc5046-6941-4882-80fb-3a0e6f6a792f service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:30.220832 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-e613d43f-9024-4691-8aa8-76555cf6e51e tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 200 len: 1510 microversion: 2.1 time: 0.200161 Aug 07 01:50:30.221148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c7dce2b3-1e67-4014-be36-64b873445fc1 req-41bc5046-6941-4882-80fb-3a0e6f6a792f service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:30.221813 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1258/2247] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:50:30 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 1510 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:50:30.222124 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-c7dce2b3-1e67-4014-be36-64b873445fc1 req-41bc5046-6941-4882-80fb-3a0e6f6a792f service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:30.248906 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-c7dce2b3-1e67-4014-be36-64b873445fc1 req-41bc5046-6941-4882-80fb-3a0e6f6a792f service nova] Creating event network-vif-deleted:7fe8e7c8-c5ab-41e4-9fdb-393c265c2479 for instance 65635777-b502-4848-8327-c39ccd195e5d on hv-cinder-maste Aug 07 01:50:30.255848 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-c7dce2b3-1e67-4014-be36-64b873445fc1 req-41bc5046-6941-4882-80fb-3a0e6f6a792f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058244 Aug 07 01:50:30.256692 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 990/2248] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:50:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:50:31.260474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-096e42ba-e0aa-4c0e-88ab-64b0067d1ede tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:50:31.261574 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-096e42ba-e0aa-4c0e-88ab-64b0067d1ede tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] [instance: 65635777-b502-4848-8327-c39ccd195e5d] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:50:31.273478 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-096e42ba-e0aa-4c0e-88ab-64b0067d1ede tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:31.274310 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-096e42ba-e0aa-4c0e-88ab-64b0067d1ede tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:31.275100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-096e42ba-e0aa-4c0e-88ab-64b0067d1ede tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:31.276417 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-096e42ba-e0aa-4c0e-88ab-64b0067d1ede tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:50:31.277333 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-096e42ba-e0aa-4c0e-88ab-64b0067d1ede tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:50:31.278090 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-096e42ba-e0aa-4c0e-88ab-64b0067d1ede tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:50:31.305633 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.wsgi [None req-096e42ba-e0aa-4c0e-88ab-64b0067d1ede tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] HTTP exception thrown: Instance 65635777-b502-4848-8327-c39ccd195e5d could not be found. Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-096e42ba-e0aa-4c0e-88ab-64b0067d1ede tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Request method failure captured: Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: request: GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d HTTP/1.1 Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept: application/json Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Accept-Encoding: identity Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Connection: close Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Length: 0 Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Content-Type: application/json Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Host: 10.222.0.26 Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Openstack-System-Scope: None Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: User-Agent: python-urllib3/1.26.11 Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Auth-Token: gAAAAABi7xnPQSQsbHsL2SrVh9srh6F1pP5uHu5BFqaR2gUCkjKkPA8aUQ0lvE0puU1GDrZedNiBYbfTh0IAvzL8zy3w14tC4TpETWq7hGnm-b-hyulT7b5SABfBOapJ5fZEcYgC-O8twlz_rhbX3pUkxAn-4TWafkOqE4Ai6PfTkYdYqMU89Mc Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Id: None Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Domain-Name: None Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Identity-Status: Confirmed Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Is-Admin-Project: True Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Id: default Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Domain-Name: Default Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Id: 673afe3c625d419e89fe80f3db4574fb Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Project-Name: tempest-TestMinimumBasicScenario-301244876 Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Role: member,reader Aug 07 01:50:31.308715 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Roles: member,reader Aug 07 01:50:31.310105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/673afe3c625d419e89fe80f3db4574fb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/673afe3c625d419e89fe80f3db4574fb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_673afe3c625d419e89fe80f3db4574fb"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/673afe3c625d419e89fe80f3db4574fb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:50:31.310105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant: tempest-TestMinimumBasicScenario-301244876 Aug 07 01:50:31.310105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Id: 673afe3c625d419e89fe80f3db4574fb Aug 07 01:50:31.310105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-Tenant-Name: tempest-TestMinimumBasicScenario-301244876 Aug 07 01:50:31.310105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User: tempest-TestMinimumBasicScenario-301244876-project Aug 07 01:50:31.310105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Id: default Aug 07 01:50:31.310105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Domain-Name: Default Aug 07 01:50:31.310105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Id: a11fb6edaae94f8ba2214e5ec0c65972 Aug 07 01:50:31.310105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: X-User-Name: tempest-TestMinimumBasicScenario-301244876-project Aug 07 01:50:31.310105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: method: > Aug 07 01:50:31.310105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: exception: Instance 65635777-b502-4848-8327-c39ccd195e5d could not be found. Aug 07 01:50:31.310105 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: action_args: {'id': '65635777-b502-4848-8327-c39ccd195e5d'} Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 65635777-b502-4848-8327-c39ccd195e5d could not be found. Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:50:31.310817 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 65635777-b502-4848-8327-c39ccd195e5d could not be found. Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 65635777-b502-4848-8327-c39ccd195e5d could not be found. Aug 07 01:50:31.312031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR nova.api.openstack.wsgi Aug 07 01:50:31.313013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-096e42ba-e0aa-4c0e-88ab-64b0067d1ede tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Returning 404 to user: Instance 65635777-b502-4848-8327-c39ccd195e5d could not be found. {{(pid=117983) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:50:31.313013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-096e42ba-e0aa-4c0e-88ab-64b0067d1ede tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d" status: 404 len: 111 microversion: 2.1 time: 0.057927 Aug 07 01:50:31.313013 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1259/2249] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:50:31 2022] GET /compute/v2.1/servers/65635777-b502-4848-8327-c39ccd195e5d => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:50:31.323047 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f3674f41-e839-4fa0-9cef-32db3bd5e1ef tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] Calling method '.version_select at 0x7f1fad146550>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:50:31.345615 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f3674f41-e839-4fa0-9cef-32db3bd5e1ef tempest-TestMinimumBasicScenario-301244876 tempest-TestMinimumBasicScenario-301244876-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-115664317" status: 202 len: 0 microversion: 2.1 time: 0.027818 Aug 07 01:50:31.346214 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 991/2250] 10.222.0.26 () {58 vars in 1269 bytes} [Sun Aug 7 01:50:31 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-115664317 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:51:09.794050 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f3540566-e450-4eee-bbad-fc592fc28bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:51:09.794767 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f3540566-e450-4eee-bbad-fc592fc28bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:09.797100 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=14 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:51:09.797761 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=30 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:51:09.803997 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f3540566-e450-4eee-bbad-fc592fc28bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:09.804512 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f3540566-e450-4eee-bbad-fc592fc28bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:09.804953 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f3540566-e450-4eee-bbad-fc592fc28bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:09.805669 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f3540566-e450-4eee-bbad-fc592fc28bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:09.806128 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f3540566-e450-4eee-bbad-fc592fc28bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:09.806552 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f3540566-e450-4eee-bbad-fc592fc28bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:09.845149 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.objects.instance [None req-f3540566-e450-4eee-bbad-fc592fc28bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lazy-loading 'flavor' on Instance uuid 768918d9-86a2-4c55-ba2a-57f5c1dcd234 {{(pid=117983) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 07 01:51:09.945569 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f3540566-e450-4eee-bbad-fc592fc28bc7 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "POST /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234/action" status: 202 len: 0 microversion: 2.1 time: 0.155351 Aug 07 01:51:09.946955 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1260/2251] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:51:09 2022] POST /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234/action => generated 0 bytes in 158 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 07 01:51:09.956624 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=41,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:51:09.958150 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=20,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:51:09.971981 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-ab0358ee-ef8c-4a3a-8417-b8c77e6ef90d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:09.973120 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-ab0358ee-ef8c-4a3a-8417-b8c77e6ef90d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:09.984524 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab0358ee-ef8c-4a3a-8417-b8c77e6ef90d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:09.985448 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab0358ee-ef8c-4a3a-8417-b8c77e6ef90d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:09.986275 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab0358ee-ef8c-4a3a-8417-b8c77e6ef90d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:09.987886 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab0358ee-ef8c-4a3a-8417-b8c77e6ef90d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:09.988972 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab0358ee-ef8c-4a3a-8417-b8c77e6ef90d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:09.989772 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-ab0358ee-ef8c-4a3a-8417-b8c77e6ef90d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:10.180127 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-ab0358ee-ef8c-4a3a-8417-b8c77e6ef90d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:10.188648 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-ab0358ee-ef8c-4a3a-8417-b8c77e6ef90d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1570 microversion: 2.1 time: 0.235808 Aug 07 01:51:10.189246 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 992/2252] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:09 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1570 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:51:11.211522 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-f8b08d0c-3489-4be3-a2f9-76bbdc7de1e4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:11.212266 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-f8b08d0c-3489-4be3-a2f9-76bbdc7de1e4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:11.220122 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8b08d0c-3489-4be3-a2f9-76bbdc7de1e4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:11.220708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8b08d0c-3489-4be3-a2f9-76bbdc7de1e4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:11.221201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8b08d0c-3489-4be3-a2f9-76bbdc7de1e4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:11.221977 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8b08d0c-3489-4be3-a2f9-76bbdc7de1e4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:11.222527 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8b08d0c-3489-4be3-a2f9-76bbdc7de1e4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:11.223004 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-f8b08d0c-3489-4be3-a2f9-76bbdc7de1e4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:11.380769 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-f8b08d0c-3489-4be3-a2f9-76bbdc7de1e4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:11.389996 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-f8b08d0c-3489-4be3-a2f9-76bbdc7de1e4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1570 microversion: 2.1 time: 0.182888 Aug 07 01:51:11.390754 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1261/2253] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:11 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1570 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:51:12.413047 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-7417523e-5b38-487b-99ba-f511e30b0fc0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:12.414030 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-7417523e-5b38-487b-99ba-f511e30b0fc0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:12.421465 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7417523e-5b38-487b-99ba-f511e30b0fc0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:12.421958 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7417523e-5b38-487b-99ba-f511e30b0fc0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:12.422392 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7417523e-5b38-487b-99ba-f511e30b0fc0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:12.423110 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7417523e-5b38-487b-99ba-f511e30b0fc0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:12.423562 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7417523e-5b38-487b-99ba-f511e30b0fc0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:12.423986 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-7417523e-5b38-487b-99ba-f511e30b0fc0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:12.572183 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-7417523e-5b38-487b-99ba-f511e30b0fc0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:12.580907 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-7417523e-5b38-487b-99ba-f511e30b0fc0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1570 microversion: 2.1 time: 0.171822 Aug 07 01:51:12.581687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 993/2254] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:12 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1570 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:51:13.602916 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4701a4ce-1e9e-42bb-bda7-20c9caa802cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:13.603782 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4701a4ce-1e9e-42bb-bda7-20c9caa802cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:13.611210 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4701a4ce-1e9e-42bb-bda7-20c9caa802cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:13.611807 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4701a4ce-1e9e-42bb-bda7-20c9caa802cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:13.612337 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4701a4ce-1e9e-42bb-bda7-20c9caa802cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:13.613120 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4701a4ce-1e9e-42bb-bda7-20c9caa802cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:13.613627 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4701a4ce-1e9e-42bb-bda7-20c9caa802cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:13.614113 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4701a4ce-1e9e-42bb-bda7-20c9caa802cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:14.057074 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4701a4ce-1e9e-42bb-bda7-20c9caa802cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:14.066772 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4701a4ce-1e9e-42bb-bda7-20c9caa802cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1570 microversion: 2.1 time: 0.467966 Aug 07 01:51:14.067370 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1262/2255] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:13 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1570 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:51:15.089353 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-9bd2bdbe-050c-4b04-8436-42ab373df549 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:15.090217 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-9bd2bdbe-050c-4b04-8436-42ab373df549 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:15.096831 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9bd2bdbe-050c-4b04-8436-42ab373df549 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:15.097378 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9bd2bdbe-050c-4b04-8436-42ab373df549 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:15.097861 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9bd2bdbe-050c-4b04-8436-42ab373df549 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:15.098625 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9bd2bdbe-050c-4b04-8436-42ab373df549 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:15.099121 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9bd2bdbe-050c-4b04-8436-42ab373df549 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:15.099595 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-9bd2bdbe-050c-4b04-8436-42ab373df549 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:15.255443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-9bd2bdbe-050c-4b04-8436-42ab373df549 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:15.265475 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-9bd2bdbe-050c-4b04-8436-42ab373df549 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1570 microversion: 2.1 time: 0.179877 Aug 07 01:51:15.267148 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 994/2256] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:15 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1570 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:51:16.295174 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-d1f83f4c-eb7f-41cd-ad13-ffe571051d7d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:16.296024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-d1f83f4c-eb7f-41cd-ad13-ffe571051d7d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:16.304140 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1f83f4c-eb7f-41cd-ad13-ffe571051d7d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:16.304714 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1f83f4c-eb7f-41cd-ad13-ffe571051d7d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:16.305161 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1f83f4c-eb7f-41cd-ad13-ffe571051d7d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:16.305907 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1f83f4c-eb7f-41cd-ad13-ffe571051d7d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:16.306374 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1f83f4c-eb7f-41cd-ad13-ffe571051d7d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:16.306808 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-d1f83f4c-eb7f-41cd-ad13-ffe571051d7d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:16.509201 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-d1f83f4c-eb7f-41cd-ad13-ffe571051d7d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:16.518169 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-d1f83f4c-eb7f-41cd-ad13-ffe571051d7d tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1570 microversion: 2.1 time: 0.227848 Aug 07 01:51:16.518752 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1263/2257] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:16 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1570 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:51:17.550350 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-bc05bac0-b91a-4a96-be5b-08117906148e tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:17.551116 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-bc05bac0-b91a-4a96-be5b-08117906148e tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:17.557992 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc05bac0-b91a-4a96-be5b-08117906148e tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:17.558478 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc05bac0-b91a-4a96-be5b-08117906148e tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:17.558912 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc05bac0-b91a-4a96-be5b-08117906148e tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:17.559626 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc05bac0-b91a-4a96-be5b-08117906148e tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:17.560080 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc05bac0-b91a-4a96-be5b-08117906148e tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:17.560551 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-bc05bac0-b91a-4a96-be5b-08117906148e tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:17.727870 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-bc05bac0-b91a-4a96-be5b-08117906148e tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:17.737223 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-bc05bac0-b91a-4a96-be5b-08117906148e tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1570 microversion: 2.1 time: 0.190982 Aug 07 01:51:17.738276 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 995/2258] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:17 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1570 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:51:18.766027 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-6f19529c-f64d-4b66-9c62-e366b88184cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:18.766870 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-6f19529c-f64d-4b66-9c62-e366b88184cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:18.773853 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f19529c-f64d-4b66-9c62-e366b88184cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:18.774337 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f19529c-f64d-4b66-9c62-e366b88184cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:18.774773 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f19529c-f64d-4b66-9c62-e366b88184cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:18.775488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f19529c-f64d-4b66-9c62-e366b88184cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:18.775940 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f19529c-f64d-4b66-9c62-e366b88184cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:18.776410 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-6f19529c-f64d-4b66-9c62-e366b88184cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:18.931059 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-6f19529c-f64d-4b66-9c62-e366b88184cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:18.940763 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-6f19529c-f64d-4b66-9c62-e366b88184cd tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1570 microversion: 2.1 time: 0.178562 Aug 07 01:51:18.941488 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1264/2259] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:18 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1570 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:51:19.969986 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-559890ab-9470-4c43-8805-0a1e3f4844b4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:19.970634 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-559890ab-9470-4c43-8805-0a1e3f4844b4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:19.977936 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-559890ab-9470-4c43-8805-0a1e3f4844b4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:19.978747 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-559890ab-9470-4c43-8805-0a1e3f4844b4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:19.979270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-559890ab-9470-4c43-8805-0a1e3f4844b4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:19.980032 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-559890ab-9470-4c43-8805-0a1e3f4844b4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:19.980574 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-559890ab-9470-4c43-8805-0a1e3f4844b4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:19.981637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-559890ab-9470-4c43-8805-0a1e3f4844b4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:20.133898 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-559890ab-9470-4c43-8805-0a1e3f4844b4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:20.147477 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-559890ab-9470-4c43-8805-0a1e3f4844b4 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1570 microversion: 2.1 time: 0.181802 Aug 07 01:51:20.148557 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 996/2260] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:19 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1570 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:51:21.181092 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4a9cbf5d-1646-4c53-b4dc-047bba6a0205 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:21.181783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4a9cbf5d-1646-4c53-b4dc-047bba6a0205 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:21.189416 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a9cbf5d-1646-4c53-b4dc-047bba6a0205 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:21.190002 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a9cbf5d-1646-4c53-b4dc-047bba6a0205 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:21.190494 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a9cbf5d-1646-4c53-b4dc-047bba6a0205 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:21.191255 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a9cbf5d-1646-4c53-b4dc-047bba6a0205 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:21.191756 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a9cbf5d-1646-4c53-b4dc-047bba6a0205 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:21.192228 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a9cbf5d-1646-4c53-b4dc-047bba6a0205 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:21.348828 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4a9cbf5d-1646-4c53-b4dc-047bba6a0205 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:21.361901 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4a9cbf5d-1646-4c53-b4dc-047bba6a0205 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1570 microversion: 2.1 time: 0.184546 Aug 07 01:51:21.362930 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1265/2261] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:21 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1570 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:51:22.385130 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-8ae8dc33-af83-446a-ad7f-5c1de6636f06 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:22.385818 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-8ae8dc33-af83-446a-ad7f-5c1de6636f06 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:22.393088 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ae8dc33-af83-446a-ad7f-5c1de6636f06 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:22.393586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ae8dc33-af83-446a-ad7f-5c1de6636f06 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:22.394028 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ae8dc33-af83-446a-ad7f-5c1de6636f06 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:22.394766 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ae8dc33-af83-446a-ad7f-5c1de6636f06 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:22.395226 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ae8dc33-af83-446a-ad7f-5c1de6636f06 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:22.395765 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-8ae8dc33-af83-446a-ad7f-5c1de6636f06 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:22.568465 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-8ae8dc33-af83-446a-ad7f-5c1de6636f06 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:22.582193 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-8ae8dc33-af83-446a-ad7f-5c1de6636f06 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1570 microversion: 2.1 time: 0.200832 Aug 07 01:51:22.583002 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 997/2262] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:22 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1570 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:51:23.614551 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-435de934-ead7-4560-96d0-7788ce3d6d14 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:23.615117 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-435de934-ead7-4560-96d0-7788ce3d6d14 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:23.621755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-435de934-ead7-4560-96d0-7788ce3d6d14 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:23.622253 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-435de934-ead7-4560-96d0-7788ce3d6d14 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:23.623031 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-435de934-ead7-4560-96d0-7788ce3d6d14 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:23.623820 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-435de934-ead7-4560-96d0-7788ce3d6d14 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:23.624311 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-435de934-ead7-4560-96d0-7788ce3d6d14 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:23.624757 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-435de934-ead7-4560-96d0-7788ce3d6d14 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:24.047303 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-435de934-ead7-4560-96d0-7788ce3d6d14 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:24.056474 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-435de934-ead7-4560-96d0-7788ce3d6d14 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1511 microversion: 2.1 time: 0.446101 Aug 07 01:51:24.057176 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1266/2263] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:23 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1511 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:51:24.081010 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-05a699f3-e8f5-4502-867f-c8e8138ca1bf tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:51:24.081855 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-05a699f3-e8f5-4502-867f-c8e8138ca1bf tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:24.088693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-05a699f3-e8f5-4502-867f-c8e8138ca1bf tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:24.089187 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-05a699f3-e8f5-4502-867f-c8e8138ca1bf tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:24.089619 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-05a699f3-e8f5-4502-867f-c8e8138ca1bf tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:24.091031 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-05a699f3-e8f5-4502-867f-c8e8138ca1bf tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:24.091487 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-05a699f3-e8f5-4502-867f-c8e8138ca1bf tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:24.091914 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-05a699f3-e8f5-4502-867f-c8e8138ca1bf tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:24.144707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-05a699f3-e8f5-4502-867f-c8e8138ca1bf tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=117984) unshelve /opt/stack/nova/nova/compute/api.py:4531}} Aug 07 01:51:24.229229 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-05a699f3-e8f5-4502-867f-c8e8138ca1bf tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "POST /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234/action" status: 202 len: 0 microversion: 2.1 time: 0.151747 Aug 07 01:51:24.230377 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 998/2264] 10.222.0.26 () {60 vars in 1278 bytes} [Sun Aug 7 01:51:24 2022] POST /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234/action => generated 0 bytes in 154 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 07 01:51:24.240541 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-4a20aff4-2d33-4f54-858f-59b93cbd7fac tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:24.241538 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-4a20aff4-2d33-4f54-858f-59b93cbd7fac tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:24.249395 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a20aff4-2d33-4f54-858f-59b93cbd7fac tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:24.249991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a20aff4-2d33-4f54-858f-59b93cbd7fac tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:24.250445 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a20aff4-2d33-4f54-858f-59b93cbd7fac tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:24.251184 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a20aff4-2d33-4f54-858f-59b93cbd7fac tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:24.251652 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a20aff4-2d33-4f54-858f-59b93cbd7fac tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:24.252086 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-4a20aff4-2d33-4f54-858f-59b93cbd7fac tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:24.475110 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-4a20aff4-2d33-4f54-858f-59b93cbd7fac tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:24.489102 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-4a20aff4-2d33-4f54-858f-59b93cbd7fac tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1519 microversion: 2.1 time: 0.252946 Aug 07 01:51:24.490220 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1267/2265] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:24 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1519 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:51:25.306234 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-fb4d7dc4-6392-4a86-b2e8-580732d97046 req-5c0fa528-4655-4ce5-8775-75cbeff67c6f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "768918d9-86a2-4c55-ba2a-57f5c1dcd234", "name": "network-vif-unplugged", "status": "completed", "tag": "ebb91259-4e43-449e-b5bb-c646943874ee"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:51:25.327410 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-fb4d7dc4-6392-4a86-b2e8-580732d97046 req-5c0fa528-4655-4ce5-8775-75cbeff67c6f service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:25.328681 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-fb4d7dc4-6392-4a86-b2e8-580732d97046 req-5c0fa528-4655-4ce5-8775-75cbeff67c6f service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:25.329565 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-fb4d7dc4-6392-4a86-b2e8-580732d97046 req-5c0fa528-4655-4ce5-8775-75cbeff67c6f service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:25.355426 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.compute.server_external_events [req-fb4d7dc4-6392-4a86-b2e8-580732d97046 req-5c0fa528-4655-4ce5-8775-75cbeff67c6f service nova] Unable to find a host for instance 768918d9-86a2-4c55-ba2a-57f5c1dcd234. Dropping event network-vif-unplugged {{(pid=117984) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Aug 07 01:51:25.357887 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-fb4d7dc4-6392-4a86-b2e8-580732d97046 req-5c0fa528-4655-4ce5-8775-75cbeff67c6f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.058839 Aug 07 01:51:25.359109 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 999/2266] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:51:25 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 61 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Aug 07 01:51:25.524582 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-5586fd2a-863b-456c-8810-5879a30bad99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:25.525222 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-5586fd2a-863b-456c-8810-5879a30bad99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:25.532093 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5586fd2a-863b-456c-8810-5879a30bad99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:25.532695 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5586fd2a-863b-456c-8810-5879a30bad99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:25.533223 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5586fd2a-863b-456c-8810-5879a30bad99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:25.533988 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5586fd2a-863b-456c-8810-5879a30bad99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:25.534444 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5586fd2a-863b-456c-8810-5879a30bad99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:25.534871 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-5586fd2a-863b-456c-8810-5879a30bad99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:25.706811 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-5586fd2a-863b-456c-8810-5879a30bad99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:25.721188 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-5586fd2a-863b-456c-8810-5879a30bad99 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1521 microversion: 2.1 time: 0.200281 Aug 07 01:51:25.722203 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1268/2267] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:25 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1521 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:51:26.756113 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-25b94db5-2a2a-464e-85b4-0b3349800fa0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:26.757244 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-25b94db5-2a2a-464e-85b4-0b3349800fa0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:26.766255 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-25b94db5-2a2a-464e-85b4-0b3349800fa0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:26.766899 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-25b94db5-2a2a-464e-85b4-0b3349800fa0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:26.767505 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-25b94db5-2a2a-464e-85b4-0b3349800fa0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:26.768444 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-25b94db5-2a2a-464e-85b4-0b3349800fa0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:26.769087 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-25b94db5-2a2a-464e-85b4-0b3349800fa0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:26.769720 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-25b94db5-2a2a-464e-85b4-0b3349800fa0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:27.020886 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-25b94db5-2a2a-464e-85b4-0b3349800fa0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:27.031627 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-25b94db5-2a2a-464e-85b4-0b3349800fa0 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1577 microversion: 2.1 time: 0.279955 Aug 07 01:51:27.032404 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 1000/2268] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:26 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1577 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:51:28.056972 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-7a81a580-404d-413f-933a-28f26b2dc6b1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:28.057714 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-7a81a580-404d-413f-933a-28f26b2dc6b1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:28.065357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7a81a580-404d-413f-933a-28f26b2dc6b1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:28.065902 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7a81a580-404d-413f-933a-28f26b2dc6b1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:28.066387 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7a81a580-404d-413f-933a-28f26b2dc6b1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:28.067327 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7a81a580-404d-413f-933a-28f26b2dc6b1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:28.067833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7a81a580-404d-413f-933a-28f26b2dc6b1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:28.068357 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-7a81a580-404d-413f-933a-28f26b2dc6b1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:28.220099 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-7a81a580-404d-413f-933a-28f26b2dc6b1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:28.228750 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-7a81a580-404d-413f-933a-28f26b2dc6b1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1577 microversion: 2.1 time: 0.175954 Aug 07 01:51:28.229350 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1269/2269] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:28 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1577 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:51:29.264155 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-179e2612-6818-4480-9abf-4492a9c5d1b3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:29.265024 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-179e2612-6818-4480-9abf-4492a9c5d1b3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:29.272752 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-179e2612-6818-4480-9abf-4492a9c5d1b3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:29.273320 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-179e2612-6818-4480-9abf-4492a9c5d1b3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:29.273845 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-179e2612-6818-4480-9abf-4492a9c5d1b3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:29.274898 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-179e2612-6818-4480-9abf-4492a9c5d1b3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:29.275467 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-179e2612-6818-4480-9abf-4492a9c5d1b3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:29.275988 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-179e2612-6818-4480-9abf-4492a9c5d1b3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:29.435528 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-179e2612-6818-4480-9abf-4492a9c5d1b3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:29.443857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-179e2612-6818-4480-9abf-4492a9c5d1b3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1577 microversion: 2.1 time: 0.184477 Aug 07 01:51:29.444498 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 1001/2270] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:29 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1577 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:51:30.476841 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-9299e28e-2e15-4bde-9257-5f491379265f tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:30.477470 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-9299e28e-2e15-4bde-9257-5f491379265f tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:30.485214 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9299e28e-2e15-4bde-9257-5f491379265f tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:30.485708 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9299e28e-2e15-4bde-9257-5f491379265f tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:30.486150 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9299e28e-2e15-4bde-9257-5f491379265f tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:30.486880 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9299e28e-2e15-4bde-9257-5f491379265f tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:30.487337 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9299e28e-2e15-4bde-9257-5f491379265f tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:30.487798 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-9299e28e-2e15-4bde-9257-5f491379265f tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:30.645835 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-9299e28e-2e15-4bde-9257-5f491379265f tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:30.655588 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-9299e28e-2e15-4bde-9257-5f491379265f tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1577 microversion: 2.1 time: 0.182521 Aug 07 01:51:30.656653 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1270/2271] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:30 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1577 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:51:31.679454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-6b6a339b-3027-49eb-be58-faa273c4d6aa tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:31.680034 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-6b6a339b-3027-49eb-be58-faa273c4d6aa tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:31.687388 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b6a339b-3027-49eb-be58-faa273c4d6aa tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:31.687864 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b6a339b-3027-49eb-be58-faa273c4d6aa tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:31.688323 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b6a339b-3027-49eb-be58-faa273c4d6aa tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:31.689025 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b6a339b-3027-49eb-be58-faa273c4d6aa tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:31.689623 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b6a339b-3027-49eb-be58-faa273c4d6aa tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:31.690048 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-6b6a339b-3027-49eb-be58-faa273c4d6aa tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:31.836896 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-6b6a339b-3027-49eb-be58-faa273c4d6aa tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:31.845181 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-6b6a339b-3027-49eb-be58-faa273c4d6aa tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1577 microversion: 2.1 time: 0.170088 Aug 07 01:51:31.845762 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 1002/2272] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:31 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1577 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:51:32.874644 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-bc43af6b-f042-4460-bbbe-5075acb4a4ca tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:32.875255 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-bc43af6b-f042-4460-bbbe-5075acb4a4ca tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:32.882428 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc43af6b-f042-4460-bbbe-5075acb4a4ca tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:32.882944 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc43af6b-f042-4460-bbbe-5075acb4a4ca tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:32.883386 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc43af6b-f042-4460-bbbe-5075acb4a4ca tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:32.884112 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc43af6b-f042-4460-bbbe-5075acb4a4ca tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:32.884597 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc43af6b-f042-4460-bbbe-5075acb4a4ca tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:32.885022 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-bc43af6b-f042-4460-bbbe-5075acb4a4ca tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:33.034750 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-bc43af6b-f042-4460-bbbe-5075acb4a4ca tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:33.043889 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-bc43af6b-f042-4460-bbbe-5075acb4a4ca tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1577 microversion: 2.1 time: 0.172578 Aug 07 01:51:33.044579 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1271/2273] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:32 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1577 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:51:34.067281 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f834a1e2-e6bb-40d6-991f-90045c4d3126 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:34.068347 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f834a1e2-e6bb-40d6-991f-90045c4d3126 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:34.076164 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f834a1e2-e6bb-40d6-991f-90045c4d3126 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:34.076688 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f834a1e2-e6bb-40d6-991f-90045c4d3126 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:34.077130 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f834a1e2-e6bb-40d6-991f-90045c4d3126 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:34.077884 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f834a1e2-e6bb-40d6-991f-90045c4d3126 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:34.078355 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f834a1e2-e6bb-40d6-991f-90045c4d3126 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:34.078790 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f834a1e2-e6bb-40d6-991f-90045c4d3126 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:34.241454 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f834a1e2-e6bb-40d6-991f-90045c4d3126 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:34.250552 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f834a1e2-e6bb-40d6-991f-90045c4d3126 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1577 microversion: 2.1 time: 0.187281 Aug 07 01:51:34.251483 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 1003/2274] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:34 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1577 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:51:35.273424 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-39378011-fcdd-4290-bf98-a94d60b07926 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:35.274376 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-39378011-fcdd-4290-bf98-a94d60b07926 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:35.283719 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-39378011-fcdd-4290-bf98-a94d60b07926 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:35.284631 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-39378011-fcdd-4290-bf98-a94d60b07926 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:35.285452 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-39378011-fcdd-4290-bf98-a94d60b07926 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:35.287236 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-39378011-fcdd-4290-bf98-a94d60b07926 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:35.287482 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-39378011-fcdd-4290-bf98-a94d60b07926 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:35.287982 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-39378011-fcdd-4290-bf98-a94d60b07926 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:35.480468 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-39378011-fcdd-4290-bf98-a94d60b07926 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:35.494668 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-39378011-fcdd-4290-bf98-a94d60b07926 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1577 microversion: 2.1 time: 0.225229 Aug 07 01:51:35.495406 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1272/2275] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:35 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1577 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:51:36.529330 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-f9d1147e-cd02-4cd9-b37e-4a57b77233b9 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:36.529945 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-f9d1147e-cd02-4cd9-b37e-4a57b77233b9 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:36.536811 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f9d1147e-cd02-4cd9-b37e-4a57b77233b9 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:36.537353 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f9d1147e-cd02-4cd9-b37e-4a57b77233b9 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:36.537849 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f9d1147e-cd02-4cd9-b37e-4a57b77233b9 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:36.538687 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f9d1147e-cd02-4cd9-b37e-4a57b77233b9 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:36.539216 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f9d1147e-cd02-4cd9-b37e-4a57b77233b9 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:36.539749 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-f9d1147e-cd02-4cd9-b37e-4a57b77233b9 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:36.687108 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-f9d1147e-cd02-4cd9-b37e-4a57b77233b9 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:36.695317 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-f9d1147e-cd02-4cd9-b37e-4a57b77233b9 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1577 microversion: 2.1 time: 0.170570 Aug 07 01:51:36.695932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 1004/2276] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:36 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1577 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:51:37.402242 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-5acfadab-2ff4-4c70-8add-12bc082c0cde req-6846c001-af40-46d7-809b-b5e76a8f3179 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "768918d9-86a2-4c55-ba2a-57f5c1dcd234", "name": "network-vif-plugged", "status": "completed", "tag": "ebb91259-4e43-449e-b5bb-c646943874ee"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:51:37.427783 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-5acfadab-2ff4-4c70-8add-12bc082c0cde req-6846c001-af40-46d7-809b-b5e76a8f3179 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:37.428691 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-5acfadab-2ff4-4c70-8add-12bc082c0cde req-6846c001-af40-46d7-809b-b5e76a8f3179 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:37.429747 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-5acfadab-2ff4-4c70-8add-12bc082c0cde req-6846c001-af40-46d7-809b-b5e76a8f3179 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:37.452068 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-5acfadab-2ff4-4c70-8add-12bc082c0cde req-6846c001-af40-46d7-809b-b5e76a8f3179 service nova] Creating event network-vif-plugged:ebb91259-4e43-449e-b5bb-c646943874ee for instance 768918d9-86a2-4c55-ba2a-57f5c1dcd234 on hv-cinder-maste Aug 07 01:51:37.466711 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-5acfadab-2ff4-4c70-8add-12bc082c0cde req-6846c001-af40-46d7-809b-b5e76a8f3179 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070650 Aug 07 01:51:37.467991 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1273/2277] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:51:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 07 01:51:37.719816 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-aea78f46-fe82-44ef-9ae1-1eb8ed706282 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:37.721334 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-aea78f46-fe82-44ef-9ae1-1eb8ed706282 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:37.729932 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aea78f46-fe82-44ef-9ae1-1eb8ed706282 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:37.730434 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aea78f46-fe82-44ef-9ae1-1eb8ed706282 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:37.731445 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aea78f46-fe82-44ef-9ae1-1eb8ed706282 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:37.732419 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aea78f46-fe82-44ef-9ae1-1eb8ed706282 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:37.732840 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aea78f46-fe82-44ef-9ae1-1eb8ed706282 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:37.733363 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-aea78f46-fe82-44ef-9ae1-1eb8ed706282 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:37.934653 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-aea78f46-fe82-44ef-9ae1-1eb8ed706282 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:37.943586 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-aea78f46-fe82-44ef-9ae1-1eb8ed706282 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1577 microversion: 2.1 time: 0.229320 Aug 07 01:51:37.944195 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 1005/2278] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:37 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1577 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:51:38.967329 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-762e1677-0054-4005-abd9-9da9b48c2c43 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:38.968037 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-762e1677-0054-4005-abd9-9da9b48c2c43 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:38.974758 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-762e1677-0054-4005-abd9-9da9b48c2c43 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:38.975169 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-762e1677-0054-4005-abd9-9da9b48c2c43 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:38.975602 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-762e1677-0054-4005-abd9-9da9b48c2c43 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:38.976332 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-762e1677-0054-4005-abd9-9da9b48c2c43 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:38.976801 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-762e1677-0054-4005-abd9-9da9b48c2c43 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:38.977211 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-762e1677-0054-4005-abd9-9da9b48c2c43 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:39.149285 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-762e1677-0054-4005-abd9-9da9b48c2c43 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:39.158378 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-762e1677-0054-4005-abd9-9da9b48c2c43 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1577 microversion: 2.1 time: 0.195689 Aug 07 01:51:39.159197 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1274/2279] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:38 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1577 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:51:40.179258 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-425041f1-4839-4749-9d93-8b9b8ad2e4c8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:51:40.179857 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-425041f1-4839-4749-9d93-8b9b8ad2e4c8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:51:40.182431 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:51:40.188840 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-425041f1-4839-4749-9d93-8b9b8ad2e4c8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:40.189371 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-425041f1-4839-4749-9d93-8b9b8ad2e4c8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:40.189810 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-425041f1-4839-4749-9d93-8b9b8ad2e4c8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:40.190594 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-425041f1-4839-4749-9d93-8b9b8ad2e4c8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:51:40.191115 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-425041f1-4839-4749-9d93-8b9b8ad2e4c8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:51:40.191611 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-425041f1-4839-4749-9d93-8b9b8ad2e4c8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:51:40.195085 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=85,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:51:40.360575 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-425041f1-4839-4749-9d93-8b9b8ad2e4c8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:51:40.369010 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-425041f1-4839-4749-9d93-8b9b8ad2e4c8 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1564 microversion: 2.1 time: 0.193685 Aug 07 01:51:40.369707 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 1006/2280] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:51:40 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1564 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:52:04.588545 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:52:04.589134 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:52:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:52:54.652881 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_api:SELECT=1 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:52:54.654166 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG dbcounter [-] [117984] Writing DB stats nova_cell1:SELECT=5 {{(pid=117984) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:52:54.658259 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-53f402c7-100e-46fd-8997-2bebce491b01 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:52:54.658833 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-53f402c7-100e-46fd-8997-2bebce491b01 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:52:54.664167 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [req-ae2ca163-d4f9-4b57-992d-9b44dc96b812 req-14090c79-4c95-43fa-ac5d-75e750b8890c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "768918d9-86a2-4c55-ba2a-57f5c1dcd234", "tag": "ebb91259-4e43-449e-b5bb-c646943874ee"}]} {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:52:54.674371 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_api:SELECT=30 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:52:54.675588 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG dbcounter [-] [117983] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117983) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 01:52:54.680793 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53f402c7-100e-46fd-8997-2bebce491b01 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:52:54.681557 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53f402c7-100e-46fd-8997-2bebce491b01 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:52:54.682067 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-ae2ca163-d4f9-4b57-992d-9b44dc96b812 req-14090c79-4c95-43fa-ac5d-75e750b8890c service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:52:54.682345 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53f402c7-100e-46fd-8997-2bebce491b01 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:52:54.682597 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-ae2ca163-d4f9-4b57-992d-9b44dc96b812 req-14090c79-4c95-43fa-ac5d-75e750b8890c service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:52:54.683047 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [req-ae2ca163-d4f9-4b57-992d-9b44dc96b812 req-14090c79-4c95-43fa-ac5d-75e750b8890c service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:52:54.683534 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53f402c7-100e-46fd-8997-2bebce491b01 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:52:54.684162 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53f402c7-100e-46fd-8997-2bebce491b01 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:52:54.684765 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-53f402c7-100e-46fd-8997-2bebce491b01 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:52:54.695933 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.compute.server_external_events [req-ae2ca163-d4f9-4b57-992d-9b44dc96b812 req-14090c79-4c95-43fa-ac5d-75e750b8890c service nova] Creating event network-changed:ebb91259-4e43-449e-b5bb-c646943874ee for instance 768918d9-86a2-4c55-ba2a-57f5c1dcd234 on hv-cinder-maste Aug 07 01:52:54.702102 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [req-ae2ca163-d4f9-4b57-992d-9b44dc96b812 req-14090c79-4c95-43fa-ac5d-75e750b8890c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050453 Aug 07 01:52:54.702998 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 1007/2281] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:52:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:52:54.717048 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-53f402c7-100e-46fd-8997-2bebce491b01 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Going to try to terminate instance {{(pid=117983) delete /opt/stack/nova/nova/compute/api.py:2666}} Aug 07 01:52:54.818935 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-53f402c7-100e-46fd-8997-2bebce491b01 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "DELETE /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 204 len: 0 microversion: 2.1 time: 0.171749 Aug 07 01:52:54.819525 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1275/2282] 10.222.0.26 () {58 vars in 1239 bytes} [Sun Aug 7 01:52:54 2022] DELETE /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 07 01:52:54.829721 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-08977a29-9350-4c36-9ceb-f668f9227c90 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:52:54.830520 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-08977a29-9350-4c36-9ceb-f668f9227c90 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:52:54.839219 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-08977a29-9350-4c36-9ceb-f668f9227c90 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:52:54.839980 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-08977a29-9350-4c36-9ceb-f668f9227c90 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:52:54.840457 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-08977a29-9350-4c36-9ceb-f668f9227c90 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:52:54.841371 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-08977a29-9350-4c36-9ceb-f668f9227c90 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:52:54.841837 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-08977a29-9350-4c36-9ceb-f668f9227c90 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:52:54.842270 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-08977a29-9350-4c36-9ceb-f668f9227c90 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:52:55.032603 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-08977a29-9350-4c36-9ceb-f668f9227c90 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:52:55.042519 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-08977a29-9350-4c36-9ceb-f668f9227c90 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1570 microversion: 2.1 time: 0.217592 Aug 07 01:52:55.043443 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 1008/2283] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:52:54 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1570 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:52:56.068712 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-a6bb72f8-8194-4b00-92de-5de618fa1576 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:52:56.069755 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.compute.api [None req-a6bb72f8-8194-4b00-92de-5de618fa1576 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117983) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:52:56.077728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6bb72f8-8194-4b00-92de-5de618fa1576 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:52:56.078224 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6bb72f8-8194-4b00-92de-5de618fa1576 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:52:56.078667 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6bb72f8-8194-4b00-92de-5de618fa1576 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:52:56.079427 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6bb72f8-8194-4b00-92de-5de618fa1576 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:52:56.079923 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6bb72f8-8194-4b00-92de-5de618fa1576 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:52:56.080382 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [None req-a6bb72f8-8194-4b00-92de-5de618fa1576 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:52:56.260532 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.policy [None req-a6bb72f8-8194-4b00-92de-5de618fa1576 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117983) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:52:56.268367 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-a6bb72f8-8194-4b00-92de-5de618fa1576 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1570 microversion: 2.1 time: 0.206360 Aug 07 01:52:56.268946 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1276/2284] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:52:56 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1570 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 07 01:52:57.293501 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-cbdec784-2c27-4feb-8ee1-320ef817d1a1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:52:57.294348 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-cbdec784-2c27-4feb-8ee1-320ef817d1a1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:52:57.301368 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbdec784-2c27-4feb-8ee1-320ef817d1a1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:52:57.301901 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbdec784-2c27-4feb-8ee1-320ef817d1a1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:52:57.302338 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbdec784-2c27-4feb-8ee1-320ef817d1a1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:52:57.303057 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbdec784-2c27-4feb-8ee1-320ef817d1a1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:52:57.303514 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbdec784-2c27-4feb-8ee1-320ef817d1a1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:52:57.303938 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-cbdec784-2c27-4feb-8ee1-320ef817d1a1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:52:57.421728 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.policy [None req-cbdec784-2c27-4feb-8ee1-320ef817d1a1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3059ad20d01b49adbb6dbb8ca75e451c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3df9bc7e5f54e938a0ab2bdc841ecd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117984) authorize /opt/stack/nova/nova/policy.py:199}} Aug 07 01:52:57.429759 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-cbdec784-2c27-4feb-8ee1-320ef817d1a1 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 200 len: 1383 microversion: 2.1 time: 0.140218 Aug 07 01:52:57.430343 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 1009/2285] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:52:57 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 1383 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 07 01:52:57.532374 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [req-764de8e0-c9ff-4715-9db6-93ed6cdb0b8e req-ce3a89b6-983a-46a1-8e6f-38ddd3ecd4f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "768918d9-86a2-4c55-ba2a-57f5c1dcd234", "name": "network-vif-deleted", "tag": "ebb91259-4e43-449e-b5bb-c646943874ee"}]} {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 07 01:52:57.542255 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-764de8e0-c9ff-4715-9db6-93ed6cdb0b8e req-ce3a89b6-983a-46a1-8e6f-38ddd3ecd4f4 service nova] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:52:57.543728 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-764de8e0-c9ff-4715-9db6-93ed6cdb0b8e req-ce3a89b6-983a-46a1-8e6f-38ddd3ecd4f4 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:52:57.544176 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG oslo_concurrency.lockutils [req-764de8e0-c9ff-4715-9db6-93ed6cdb0b8e req-ce3a89b6-983a-46a1-8e6f-38ddd3ecd4f4 service nova] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117983) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:52:57.557043 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.compute.server_external_events [req-764de8e0-c9ff-4715-9db6-93ed6cdb0b8e req-ce3a89b6-983a-46a1-8e6f-38ddd3ecd4f4 service nova] Creating event network-vif-deleted:ebb91259-4e43-449e-b5bb-c646943874ee for instance 768918d9-86a2-4c55-ba2a-57f5c1dcd234 on hv-cinder-maste Aug 07 01:52:57.562576 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [req-764de8e0-c9ff-4715-9db6-93ed6cdb0b8e req-ce3a89b6-983a-46a1-8e6f-38ddd3ecd4f4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.106297 Aug 07 01:52:57.563219 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1277/2286] 10.222.0.26 () {64 vars in 1317 bytes} [Sun Aug 7 01:52:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 07 01:52:58.466773 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-62ed0226-73fe-4e1e-8cca-1e49a8fac3f3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '>' {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:52:58.467891 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.compute.api [None req-62ed0226-73fe-4e1e-8cca-1e49a8fac3f3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [instance: 768918d9-86a2-4c55-ba2a-57f5c1dcd234] Fetching instance by UUID {{(pid=117984) get /opt/stack/nova/nova/compute/api.py:2922}} Aug 07 01:52:58.481600 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62ed0226-73fe-4e1e-8cca-1e49a8fac3f3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:52:58.482458 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62ed0226-73fe-4e1e-8cca-1e49a8fac3f3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:52:58.483289 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62ed0226-73fe-4e1e-8cca-1e49a8fac3f3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:52:58.484693 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62ed0226-73fe-4e1e-8cca-1e49a8fac3f3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Acquiring lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 07 01:52:58.485591 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62ed0226-73fe-4e1e-8cca-1e49a8fac3f3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 07 01:52:58.486083 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG oslo_concurrency.lockutils [None req-62ed0226-73fe-4e1e-8cca-1e49a8fac3f3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Lock "dfea601e-2ab8-4cc9-8a98-b642673a3d20" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117984) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 07 01:52:58.502448 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.wsgi [None req-62ed0226-73fe-4e1e-8cca-1e49a8fac3f3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] HTTP exception thrown: Instance 768918d9-86a2-4c55-ba2a-57f5c1dcd234 could not be found. Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-62ed0226-73fe-4e1e-8cca-1e49a8fac3f3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Request method failure captured: Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: request: GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 HTTP/1.1 Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept: application/json Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Accept-Encoding: identity Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Connection: close Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Length: 0 Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Content-Type: application/json Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Host: 10.222.0.26 Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Openstack-System-Scope: None Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: User-Agent: python-urllib3/1.26.11 Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Auth-Token: gAAAAABi7xneOC9NS94LKWuBmiaCK_38OzTGzZr6InBk7tZEburzhi2Cz62f5juB42PYw0AHHhn6YSpHtcDA4DGoHwo5nJQ8xIKsddYrDNugUBZHZQSQOOzUHfMM-XqE4mQphUNEQK7z6Y25orSeB8oPT-kega5tJ5f10WkmXCyS8om5_BZpiBQ Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Id: None Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Domain-Name: None Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Identity-Status: Confirmed Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Is-Admin-Project: True Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Id: default Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Domain-Name: Default Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Id: a3df9bc7e5f54e938a0ab2bdc841ecd3 Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Project-Name: tempest-TestShelveInstance-1072181000 Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Role: member,reader Aug 07 01:52:58.504179 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Roles: member,reader Aug 07 01:52:58.505637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Service-Catalog: [{"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a3df9bc7e5f54e938a0ab2bdc841ecd3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a3df9bc7e5f54e938a0ab2bdc841ecd3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a3df9bc7e5f54e938a0ab2bdc841ecd3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a3df9bc7e5f54e938a0ab2bdc841ecd3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}] Aug 07 01:52:58.505637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant: tempest-TestShelveInstance-1072181000 Aug 07 01:52:58.505637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Id: a3df9bc7e5f54e938a0ab2bdc841ecd3 Aug 07 01:52:58.505637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-Tenant-Name: tempest-TestShelveInstance-1072181000 Aug 07 01:52:58.505637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User: tempest-TestShelveInstance-1072181000-project Aug 07 01:52:58.505637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Id: default Aug 07 01:52:58.505637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Domain-Name: Default Aug 07 01:52:58.505637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Id: 3059ad20d01b49adbb6dbb8ca75e451c Aug 07 01:52:58.505637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: X-User-Name: tempest-TestShelveInstance-1072181000-project Aug 07 01:52:58.505637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: method: > Aug 07 01:52:58.505637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: exception: Instance 768918d9-86a2-4c55-ba2a-57f5c1dcd234 could not be found. Aug 07 01:52:58.505637 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: action_args: {'id': '768918d9-86a2-4c55-ba2a-57f5c1dcd234'} Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: {{(pid=117984) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 768918d9-86a2-4c55-ba2a-57f5c1dcd234 could not be found. Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 07 01:52:58.506438 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 768918d9-86a2-4c55-ba2a-57f5c1dcd234 could not be found. Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 768918d9-86a2-4c55-ba2a-57f5c1dcd234 could not be found. Aug 07 01:52:58.507618 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: ERROR nova.api.openstack.wsgi Aug 07 01:52:58.508865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: DEBUG nova.api.openstack.wsgi [None req-62ed0226-73fe-4e1e-8cca-1e49a8fac3f3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Returning 404 to user: Instance 768918d9-86a2-4c55-ba2a-57f5c1dcd234 could not be found. {{(pid=117984) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 07 01:52:58.508865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO nova.api.openstack.requestlog [None req-62ed0226-73fe-4e1e-8cca-1e49a8fac3f3 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234" status: 404 len: 111 microversion: 2.1 time: 0.045545 Aug 07 01:52:58.508865 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: [pid: 117984|app: 0|req: 1010/2287] 10.222.0.26 () {58 vars in 1236 bytes} [Sun Aug 7 01:52:58 2022] GET /compute/v2.1/servers/768918d9-86a2-4c55-ba2a-57f5c1dcd234 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 07 01:52:58.730024 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: DEBUG nova.api.openstack.wsgi [None req-b3f722ab-eebe-4928-a28a-6ca25db59b27 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] Calling method '.version_select at 0x7f1fad3abdc0>' {{(pid=117983) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 07 01:52:58.736133 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: ERROR oslo.messaging._drivers.impl_rabbit [None req-b3f722ab-eebe-4928-a28a-6ca25db59b27 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [eafb0ecb-6d84-44f7-80a1-140e7b7b2f53] AMQP server on 10.222.0.26:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Aug 07 01:52:59.745740 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [None req-b3f722ab-eebe-4928-a28a-6ca25db59b27 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] [eafb0ecb-6d84-44f7-80a1-140e7b7b2f53] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 44816. Aug 07 01:52:59.768180 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO nova.api.openstack.requestlog [None req-b3f722ab-eebe-4928-a28a-6ca25db59b27 tempest-TestShelveInstance-1072181000 tempest-TestShelveInstance-1072181000-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1538091592" status: 202 len: 0 microversion: 2.1 time: 1.042588 Aug 07 01:52:59.769251 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: [pid: 117983|app: 0|req: 1278/2288] 10.222.0.26 () {58 vars in 1254 bytes} [Sun Aug 7 01:52:58 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1538091592 => generated 0 bytes in 1045 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 07 01:53:28.071686 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:53:28.072696 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:53:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:55:24.598257 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:55:24.598957 dv-cinder-master-3-iscsi devstack@n-api.service[117983]: Sun Aug 7 01:55:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:55:47.013809 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:55:47.014462 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:55:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:56:48.081570 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:56:48.082299 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:56:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 07 01:57:08.512968 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 07 01:57:08.513699 dv-cinder-master-3-iscsi devstack@n-api.service[117984]: Sun Aug 7 01:57:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!